Python-ideas

python-ideas@python.org

42 participants
5083 discussions

Add support for private variables, methods and functions in Python

by Shreyan Avigyan

Private methods, functions and variables are very common in programming languages. But Python doesn't support private. It has conventions for naming so considered private but not private. Most of the time private is never required, what Python provides is more than enough. But the need for private come into place when we're dealing with passphrases and servers. For example consider this code, class A: def get(): // code to get the password self.password = password Now consider this, >>> x = A(); x.get(); x.password See what just happened? Someone just got the member variable value that the person wasn't supposed to. I suggest to add private support for functions (module __all__ methods to be more clear), methods and variables (module __all__ variables or class variables). (I very bad at reading PEPs so I may miss out something critical that's been explained already (sometimes I miss out a feature in a PEP and think about suggesting that feature when it's already there and then I realize "Oh! It's already here in this PEP"). If that's the case then please correct me.) With Regards, Shreyan Avigyan

3 hours, 58 minutes

String comprehension

by David Álvarez Lombardi

I propose a syntax for constructing/filtering strings analogous to the one available for all other builtin iterables. It could look something like this. >>> dirty = "f8sjGe7" >>> clean = c"char for char in dirty if char in string.ascii_letters" >>> clean 'fsjGe' Currently, the best way to do this (in the general case) seems to be the following. >>> clean = "".join(char for char in dirty if char in string.ascii_letters) But I think the proposed syntax would be superior for two main reasons. - Consistency with the comprehension style for all other iterables (which seems to be one of the most beloved features of python) - Confusion surrounding the str.join(iter) syntax is very well documented <https://stackoverflow.com/questions/493819/why-is-it-string-joinlist-instea…> and I believe it is particularly unintuitive when the string is empty I also believe the following reasons carry some weight. - Skips unnecessary type switching from str to iter and back to str - Much much MUCH more readable/intuitive Please let me know what you all think. It was mentioned (by @rhettinger) in the PBT issue <https://bugs.python.org/issue43900> that this will likely require a PEP which I would happily write if there is a positive response. -- *David Álvarez Lombardi* Machine Learning Spanish Linguist Amazon | Natural Language Understanding Boston, Massachusetts alvarezdqal <https://www.linkedin.com/in/alvarezdqal/>

1 day

Changing The Theme of Python Docs Site

by Abdur-Rahmaan Janhangeer

Greetings, A proposal to change the theme of the Python docs. Great softwares are not distinguished by docs crankiness and unpleasantness. Though what's more important is the content, nevertheless it can be made more enjoyable to read. The Masonite docs for example is quite nice: https://docs.masoniteproject.com/ Kind Regards, Abdur-Rahmaan Janhangeer about <https://compileralchemy.github.io/> | blog <https://www.pythonkitchen.com> github <https://github.com/Abdur-RahmaanJ> Mauritius

1 day, 1 hour

Support more conversions in format string

by Serhiy Storchaka

Currently format strings (and f-string expressions) support three conversions: !s -- str, !r -- repr and !a for ascii. I propose to add support of additional conversions: for int, float and operator.index. It will help to convert automatically printf-like format strings to f-string expressions: %d, %i, %u -- use int, %f -- use float, %o, %x -- use operator.index. For float the conversion letter is obvious -- !f. But I am not sure for what !i should be used, for int or operator.index. If make it operator.index, then !d perhaps should be used for int. If make it int, then !I perhaps should be used for operator.index. Or vice verse? Also I propose to support applying multiple conversions for the same item. It is common when you output a path or URL object as quoted string with all escaping, because in general it can contain special or non-printable characters. Currently I write f"path = {repr(str(path))}" or f"path = {str(path)!r}", but want to write f"path = {path!s!r}". Do we need support of more standard conversions? Do we want to support custom conversions (registered globally as encodings and error handlers). re.escape, html.escape and shlex.quote could be very useful in some applications.

1 day, 6 hours

Namespaces!

by Matt del Valle

Hi all! So this is a proposal for a new soft language keyword: namespace I started writing this up a few hours ago and then realized as it was starting to get away from me that there was no way this was going to be even remotely readable in email format, so I created a repo for it instead and will just link to it here. The content of the post is the README.md, which github will render for you at the following link: https://github.com/matthewgdv/namespace I'll give the TLDR form here, but I would ask that before you reply please read the full thing first, since I don't think a few bullet-points give the necessary context. You might end up bringing up points I've already addressed. It will also be very hard to see the potential benefits without seeing some actual code examples. TLDR: - In a single sentence: this proposal aims to add syntactic sugar for setting and accessing module/class/local attributes with dots in their name - the syntax for the namespace keyword is similar to the simplest form of a class definition statement (one that implicitly inherits from object), so: namespace some_name: ... # code goes here - any name bound within the namespace block is bound in exactly the same way it would be bound if the namespace block were not there, except that the namespace's name and a dot are prepended to the key when being inserted into the module/class/locals dict. - a namespace block leaves behind an object that serves to process attribute lookups on it by prepending its name plus a dot to the lookup and then delegating it to whatever object it is in scope of (module/class/locals) - This would allow for small performance wins by replacing the use of class declarations that is currently common in python for namespacing, as well as making the writer's intent explicit - Crucially, it allows for namespacing the content of classes, by grouping together related methods. This improves code clarity and is useful for library authors who design their libraries with IDE autocompletion in mind. This cannot currently be done by nesting classes. I live in the UK so I'm going to bed now (after working on this for like the last 6 hours). I'll be alive again in maybe 8 hours or so and will be able to reply to any posts here then. Cheers everyone :)

1 day, 17 hours

Add switch-case statements in Python

by Shreyan Avigyan

Most of programming languages out there have switch-case statements. Python doesn't have support for switch-case statements though. switch-case statements can sometimes be nice and tidy and sometimes it may be a horrible nightmare. But that also applies to if-elif-else statements. It would be nice to have switch-case statements support in python. Suggested syntax :- ------------------- switch expr: case expr2: case expr3: statements default: statements How should it be implemented? ----------------------------------- First of all this will require a new syntax. Second I suggest for comparing the case expr with switch expr, compare the pointers instead of objects for better performance. Usually if the ladder is represented using if-elif-else logic then it would be lower performance because we'll end up comparing objects not pointers.

2 days, 15 hours

Comprehensions within f-strings

by Valentin Berlier

Recently there's been some discussion around string comprehensions, and I wanted to look at a specific variant of the proposal in a bit more detail. Original thread: https://mail.python.org/archives/list/python-ideas@python.org/thread/MVQGP4… Let's say i have a matrix of numbers: matrix = [[randint(7, 50) / randint(1, 3) for _ in range(4)] for _ in range(4)] I want to format and display each row so that the columns are nicely lined up. Maybe also display the sum of the row at the end of each line: for row in matrix: print(''.join(f'{n:>8.3f}' for n in row) + f' | {sum(row):>8.3f}') This gives me a nicely formatted table. Now with the proposal: for row in matrix: print(f'{n for n in row:>8.3f} | {sum(row):>8.3f}') The idea is that you would be able to embed a comprehension in f-string interpolations, and that the format specifier at the end would be applied to all the generated items. This has a few advantages compared to the first version. It's a bit shorter and I find it easier to see what kind of shape the output will look like. It would also be faster since the interpreter would be able to append the formatted numbers directly to the final string. The first version needs to create a temporary string object for each number and then feed it into the iterator protocol before joining them together.

3 days, 15 hours

TACE16 text encoding for Tamil language

by பா. மு. செல்வக்குமார்

Hi, I want to use this encoding <https://github.com/vanangamudi/tace16-utf8-converter/blob/master/tace16.py>[1] for Tamil language text because it is more consistent with the language's nature, and Unicode encoding severely damages(read more here[2] <https://en.wikipedia.org/wiki/Tamil_All_Character_Encoding>) the intrinsic features of the fusion of alphabets. a specific example would be when dealing with regex. *'^[சிகு]'* is the intended expression for lines that starts with either 'சி' or 'கு' just like how in English '^[ab]' matches lines that start with either 'a' or 'b'. But since Unicode represents some of the eastern languages with multiple code points, '^[சிகு]' basically translates to '^[ச,ி,க,ு]' (using the commas for clarity) சி -> ச,ி and கு -> க,ு . Running the expression over a few words in python, gives the results as shown in the attached image. Note: expected results can be obtained by using this expression '^(சி|கு)' but this works for this specific case, but there should be a way to write expressions to match சிசிசிகுகுசிகு? regex in tamil is not a python issue. it is a unicode issue. I suppose if I can encode Tamil text in TACE16 encoding, I can use regex directly over it since (as I understand) re module runs matching over bytes. Two basic questions, 1. How do I approach writing a new text encoding codec for python and register it with the codec module. 2. How would I convert utf-8 encoded pattern for regex into the custom codec so that the pattern and input string for re.match/search is consistent. Links: [1] https://github.com/vanangamudi/tace16-utf8-converter/blob/master/tace16.py [2] https://en.wikipedia.org/wiki/Tamil_All_Character_Encoding -- Thanks, வணங்காமுடி (vanangamudi)

4 days, 18 hours

Adding str.remove()

by David Mertz

I was actually thinking about this before the recent "string comprehension" thread. I wasn't really going to post the idea, but it's similar enough that I am nudged to. Moreover, since PEP 616 added str.removeprefix() and str.removesuffix(), this feels like a natural extension of that. I find myself very often wanting to remove several substrings of similar lines to get at "the good bits" for my purpose. Log files are a good example of this, but it arises in lots of other contexts I encounter. Let's take a not-absurd hypothetical: GET [http://example.com/picture] 200 image/jpeg POST [http://nowhere.org/data] 200 application/json PUT [https://example.org/page] 200 text/html For each of these lines, I'd like to see the URL and the MIME type only. The new str.removeprefix() helps some, but not as much as I would like since the "remove a tuple of prefixes" idea was rejected for PEP 616. But even past that, very often much of what I want to remove is in the middle, not at the start or the end. I know I can use regular expressions here. However, they are definitely a higher cognitive burden, and especially so for those who haven't taught them and written about them a lot, as I have. Even for me, I'd rather not think about regexen if I don't *have to*. So probably I'll do something like this: for line in lines: for noise in ('GET', 'POST', 'PUT', '200', '[', ']'): line = line.replace(noise, '') process_line(line) That's not horrible, but it would be nicer to write: for line in lines: process_line(line.remove(('GET', 'POST', 'PUT', '200', '[', ']')) Of course, if I really needed this as much as I seem to be suggesting, I know how to write a function `remove_strings()`... and I confess I have not done that. Or at least I haven't done it in some standard "my_utils" module I always import. Nonetheless, a string method would feel even more natural than a function taking the string as an argument. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

5 days, 4 hours

Improving sys.executable for embedded Python scenarios

by Gregory Szorc

The way it works today, if you have an application embedding Python, your sys.argv[0] is (likely) your main executable and sys.executable is probably None or the empty string (per the stdlib docs which say not to set sys.executable if there isn't a path to a known `python` executable). Unfortunately, since sys.executable is a str, the executable it points to must behave as `python` does. This means that your application embedding and distributing its own Python must provide a `python` or `python`-like standalone executable and use it for sys.executable and this executable must be independent from your main application because the run-time behavior is different. (Yes, you can employ symlink hacks and your executable can sniff argv[0] and dispatch to your app or `python` accordingly. But symlinks aren't reliable on Windows and this still requires multiple files/executables.) **This limitation effectively prevents the existence of single file application binaries who also want to expose a full `python`-like environment, as there's no standard way to advertise a mechanism to invoke `python` that isn't a standalone executable with no arguments.** While applications embedding Python may not have an explicit `python` executable, they do likely have the capability to instantiate a `python`-like environment at run-time: they have the interpreter after all, they "just" need to provide a mechanism to invoke Py_RunMain() with an interpreter config initialized using the "python" profile. **I'd like to propose a long-term replacement to sys.executable that enables applications embedding Python to advertise a mechanism for invoking the same executable such that they get a `python` experience.** The easiest way to do this is to introduce a list[str] variant. Let's call it sys.python_interpreter. Here's how it would work. Say I've produced myapp.exe, a Windows application. If you run `myapp.exe python --`, the executable behaves like `python`. e.g. `myapp.exe python -- -c 'print("hello, world")'` would be equivalent to `python -c 'print("hello, world")'`. The app would set `sys.python_interpreter = ["myapp.exe", "python", "--"]`. Then Python code wanting to invoke a Python interpreter would do something like `subprocess.run(sys.python_interpreter)` and automatically dispatch through the same executable. For applications not wanting to expose a `python`-like capability, they would simply set sys.python_interpreter to None or [], just like they do with sys.executable today. In fact, I imagine Python's initialization would automatically set sys.python_interpreter to [sys.executable] by default and applications would have to opt in to a more advanced PyConfig field to make sys.python_interpreter different. This would make sys.python_interpreter behaviorally backwards compatible, so code bases could use sys.python_interpreter as a modern substitute for sys.executable, if available, without that much risk. Some applications may want more advanced mechanisms than command line arguments to dispatch off of. For example, maybe you want to key off an environment variable to activate "Python mode." This scenario is a bit harder to implement, as it would require yet another advertisement on how to invoke `python`. If subprocess had a "builder" interface for iteratively constructing a process invocation, we could expose a stdlib function to return a builder preconfigured to invoke `python`. But since such an interface doesn't exist, there's not as clean a solution for cases that require something more advanced than additional process arguments. Maybe we could make sys.python_interpreter a tuple[list[str], dict[str, str]] where that dict is environment variables to set. Doable. But I'm unconvinced the complexity is warranted, especially since the application has full control over interpreter initialization and can set most of the settings that they'd want to set through environment variables (e.g. PYTHONHOME) as part of initializing the `python`-like environment. Yes, there will be a long tail of applications needing to adapt to the reality that sys.python_interpreter exists and is a list. Checks like `if sys.executable == sys.argv[0]` will need to become more complicated. Maybe we could expose a simple "am I a Python interpreter process" in the stdlib? (The inverse "am I not a Python interpreter executable" question could also benefit from stdlib standardization, as there are unofficial mechanisms like sys.frozen and sys.meipass attempting to answer this question.) Anyway, as it stands, sys.executable just doesn't work for applications embedding Python who want to expose a full `python`-like environment from single executable distributions. I think the introduction of a new API to allow applications to "self-dispatch" to a Python interpreter could eventually lead to significant ergonomic wins for embedded Python applications. This would make Python a more attractive target for embedding, which benefits the larger Python ecosystem. Thoughts? (I rarely post here. So if this idea is actionable, please inform me of next steps to make it become a reality.)

5 days, 8 hours

Jump to page:

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas