Adding pep8-casing-compliant aliases for the entire stdlib
So I was reading the docs for the `threading` module and I stumbled upon this little note: Note: In the Python 2.x series, this module contained camelCase names for some methods and functions. These are deprecated as of Python 3.10, but they are still supported for compatibility with Python 2.5 and lower. And it got me thinking. Given that there is some precedent, would it be feasible to make a concerted effort to add aliases across the board for all public-facing stdlib types and functions that don't follow pep8-recommended casing? I realize that large chunks of the stdlib predates pep8 and therefore use various non-uniform conventions. For example, the logging module is fully camelCased, and many core types like `str` and `list` don't use PascalCase as pep8 recommends. The `collections` module is a veritable mosaic of casing conventions, with some types like `deque` and `namedtuple` being fully lowercased while others like `Counter` and `ChainMap` are PascalCased. My motivation for this twofold: 1) I'll confess that this has just always been a wart that has bothered me way more than it has any right to. I just *hate* it. Somewhere deep inside my lizard-brain it makes me unhappy to have disparate naming conventions in my code. I realize this isn't a good reason in and of itself but I wonder if this might not be the case for others as well. While I've come to accept it because that's just how it is, maybe it doesn't have to be this way? 2) It's always been an extra thing to explain when teaching python to someone. I always try to cover pep8 very early to discourage people I'm training from internalizing bad habits, and it means you have to explain that the very standard library itself contains style violations that would get flagged in most modern code reviews, and that they just have to keep in mind that despite the fact that the core language does it, they should not. So the scope of my suggestion is as follows: - lowercase types become PascalCase (e.g., `str` -> `Str`, `collections.defaultdict` -> `collections.DefaultDict`) - lowercase attributes/functions/methods become snake_case (no changes for names that only contain a single word, so `str.lower()` would be unaffected, but `str.removeprefix()` would get the alias `str.remove_prefix()`) - pep8 and the python docs are updated to state that the pep8-compliant forms of stdlib names should be strongly preferred over the legacy names, and that IDEs and linters should include (configurable?) weak warnings to discourage the use of legacy-cased stdlib names - `help()` would be special-cased for builtin types to no longer display any current non-pep8-compliant names, and the python docs would also no longer show them, instead only making a note at the top of the page as with the `threading` module. Given the horrors of the python 2.7 schism I don't think there's any rush to officially deprecate or remove the current non-pep8 names at all. I think that's the sort of thing that can happily and fully be kicked down the road. If we add aliases and they see widespread adoption to the point where the non-pep8 forms are barely ever even seen out in the wild then maybe in 10 or 20 years time when the steering council is deliberating on a new major python version they can consider rolling the removal of legacy badly-cased names into it. And if not then no big deal. So yeah, thoughts?
You're not alone—it bothers me too! I try to use them as an example of why I shouldn't obsess over all the details (a certain "hobgoblin" quote always comes to mind), but I would *always* use the more consistent version if it were to exist... On Thu, Nov 11, 2021 at 8:42 AM Matt del Valle <matthewgdv@gmail.com> wrote:
On Fri, Nov 12, 2021 at 12:41 AM Matt del Valle <matthewgdv@gmail.com> wrote:
2) It's always been an extra thing to explain when teaching python to someone. I always try to cover pep8 very early to discourage people I'm training from internalizing bad habits, and it means you have to explain that the very standard library itself contains style violations that would get flagged in most modern code reviews, and that they just have to keep in mind that despite the fact that the core language does it, they should not.
ISTM that this indicates that you're putting too much focus on PEP 8 too early. At no time does the document ever state that all Python code ever written must comply with it. New Python programmers should not feel like they're being forced into a naming convention.
So the scope of my suggestion is as follows:
- lowercase types become PascalCase (e.g., `str` -> `Str`, `collections.defaultdict` -> `collections.DefaultDict`)
Pop quiz: Which of these are types and which are functions (or something else)? bool, classmethod, divmod, enumerate, globals, map, property, sorted, super, zip collections.deque, collections.namedtuple Does it even matter? Especially: does it matter enough to force a name change if a function is replaced with a type, or vice versa?
- lowercase attributes/functions/methods become snake_case (no changes for names that only contain a single word, so `str.lower()` would be unaffected, but `str.removeprefix()` would get the alias `str.remove_prefix()`)
Are you aware that removeprefix is very new, and that the alternate name remove_prefix was rejected? https://www.python.org/dev/peps/pep-0616/#alternative-method-names
Given the horrors of the python 2.7 schism I don't think there's any rush to officially deprecate or remove the current non-pep8 names at all. I think that's the sort of thing that can happily and fully be kicked down the road.
Wait, are you deprecating them or not? I'm confused. Earlier you were saying that you clearly wanted to stop people from using the old names, but now you're saying there's no rush to deprecate them.
If we add aliases and they see widespread adoption to the point where the non-pep8 forms are barely ever even seen out in the wild then maybe in 10 or 20 years time when the steering council is deliberating on a new major python version they can consider rolling the removal of legacy badly-cased names into it. And if not then no big deal.
That will never happen. It's still possible today to find code written for older 2.x versions, including some quite popular answers on places like Stack Overflow. Suppose that these aliases were added in Python 3.11. Anyone who wants to write code compatible with 3.10 would want to continue using the existing names, and since the existing names would keep on being supported for the foreseeable future, there would be little incentive to change until several versions have passed. At that point, both names might start being used in parallel (we're talking probably 2025 or thereabouts, although it might start sooner if people are also using syntactic features from 3.11+), but it would take a VERY long time for adoption to the level that the older names are "barely ever even seen". Probably never.
So yeah, thoughts?
Absolutely no value in adding aliases for everything, especially things that can be shadowed. It's not hugely common, but suppose that you deliberately shadow the name "list" in your project - now the List alias has become disconnected from it, unless you explicitly shadow that one as well. Conversely, a much more common practice is to actually use the capitalized version as a variant: class List(list): ... This would now be shadowing just one, but not the other, of the built-ins. Confusion would abound. In closing, I'd just like to highlight one very important section of PEP 8: https://www.python.org/dev/peps/pep-0008/#a-foolish-consistency-is-the-hobgo... When a style guide becomes a boat anchor, it's not doing its job. ChrisA
That's fair enough for people learning python as a hobby or in a context that's a bit more casual than agile. I generally find myself training junior/graduate people with OOP backgrounds other than Python (Java, C#, etc.) for my data engineering team and pep8-compliance is quite important, because they won't even be able to successfully make commits unless they can pass the flake8 pre-commit hook, let alone get as far as a merge review. So I'd say that pep8 is quite important from the get-go. Pop quiz: Which of these are types and which are functions (or something
I'd argue this is actually a point *in favor* of the proposal rather than against. In going through your list I actually discovered lots of things I found extremely surprising (collections.deque is a type as I expected whereas collections.namedtuple is actually a factory function). If pep8 had been introduced at the dawn of (python-)time and the stdlib had been designed with pep8-compliance in mind, there would be no surprise at all on the day that I think to myself: 'Hmm, I want to subclass namedtuple, let's try:' class MyNamedTuple(collections.namedtuple): ... and discover that it doesn't work. Are you aware that removeprefix is very new, and that the alternate
name remove_prefix was rejected?
I was aware of that, yes. The rationale was precisely to introduce it as `str.removeprefix` for consistency with alllowercase (<- which all other considerations aside can be very unreadable sometimes, that's, I assume, why pep8 prefers snake_case) which is itself in contravention of python's official style guide. This is irrelevant in the event that this proposal is accepted. Wait, are you deprecating them or not? I'm confused. Earlier you were
saying that you clearly wanted to stop people from using the old names, but now you're saying there's no rush to deprecate them.
This is a semantic misunderstanding, I should have been clearer. In my head, 'officially deprecating' something means saying that it will *definitely* be removed in the future, and optionally specifying the precise date at which it will be removed. Since my suggestion doesn't necessarily involve ever actually removing the current names from the language (only potentially, if a future steering council decides to as part of a future proposal), I didn't consider this to be deprecating them, exactly. Hopefully that's cleared up. Sorry for the confusing wording. Suppose that these aliases were added in Python 3.11. Anyone who wants
This is an argument that can be made for any change ever made to python, and consequently I think it's an extremely weak argument. Why change anything if people won't be confident using it for at least 5 years because of compatibility concerns with current language versions? Well, as it turns out time passes and eventually that distant 5-year mark is in the past. I just checked and... wow yeah f-strings were introduced in 2016! Feels like only yesterday :) Absolutely no value in adding aliases for everything, especially
I think this is a fair point, but the same can be said of people accidentally shadowing any number of other builtins (it's probably most common with builtins like `type` or `max`). Fortunately every linter/IDE I've used has had warnings for this so it should be something that happens rarely, if ever. I'd also say that subclassing list as `List` is probably just bad style that should be discouraged anyway, because presumably if you're subclassing list you're doing it to extend it in some way, and you should pick a more descriptive name like `ChainableList` (if you're implementing a list where inplace methods return self and allow chaining, for example), rather than just `List` I'd argue that a far more common usage pattern is to assign an instance to the snake_case version of it's class name, for example: email_service = EmailService(...) or for the example in question: list = List() I think this is an entirely reasonable usage form for a hypothetical future codebase that disallows legacy-cased names and enforces it with flake8. But currently this can't be done. I think what it boils down to for me is this: If we went to a disconnected alternate universe where python had never been invented and introduced it today, in 2021, would we introduce it with a uniform naming convention, or the historical backwards-supporting mishmash of casing we've ended up with? Since I think the answer is pretty clear, I'm strongly in favor of making this minimally-invasive change that at least works towards uniform casing, even if that dizzying utopia is far beyond the horizon. Our grandchildren might thank us :p I do concede that some awkward shadowing edge-cases are the strongest argument against this proposal. I personally don't think they're that strong of an argument compared to the eventual payoff, but that's just my subjective opinion. On Thu, Nov 11, 2021 at 2:11 PM Chris Angelico <rosuav@gmail.com> wrote:
Let's just say up front that I'm a strong -1 on this proposal, as I think it is needless churn, and while it may be *technically* backward compatible, in reality it will be immensely disruptive. There's one particular point I want to pick up on, though. On Thu, 11 Nov 2021 at 15:25, Matt del Valle <matthewgdv@gmail.com> wrote:
If we went to a disconnected alternate universe where python had never been invented and introduced it today, in 2021, would we introduce it with a uniform naming convention, or the historical backwards-supporting mishmash of casing we've ended up with? Since I think the answer is pretty clear, I'm strongly in favor of making this minimally-invasive change that at least works towards uniform casing, even if that dizzying utopia is far beyond the horizon. Our grandchildren might thank us :p
Certainly, there's a lot of inconsistency that can only be justified as historical baggage. There have been a number of proposals to "tidy up" such cases, but even when focused on specific instances, the general conclusion has been that this would be too disruptive. But let's put that aside and address the broader question here. Yes, if we'd been designing everything now, we quite probably would have adopted a more consistent approach. But I think it's foolish to assume that whatever convention we defined for names would be strictly based on the type of the value. After all, even if you adopt a no-compromises stance on PEP 8 (a stance that the PEP itself rejects, by the way!) the first part of the "Naming Conventions" section says """ Names that are visible to the user as public parts of the API should follow conventions that reflect usage rather than implementation. """ To examine some specific cases, lists are a type, but list(...) is a function for constructing lists. The function-style usage is far more common than the use of list as a type name (possibly depending on how much of a static typing advocate you are...). So "list" should be lower case by that logic, and therefore according to PEP 8. And str() is a function for getting the string representation of an object as well as being a type - so should it be "str" or "Str"? That's at best a judgement call (usage is probably more evenly divided in this case), but PEP 8 supports both choices. Or to put it another way, "uniform" casing is a myth, if you read PEP 8 properly. What you actually seem to be arguing for is a renaming based on a hypothetical version of PEP 8 that is far stricter than the actual document, and which doesn't take into account the messiness of real-world APIs and applications. That's a very common, and in my opinion misguided, stance. For me, one of the best things about PEP 8 is its repeated assertions that the "rules" it defines are only guidelines and that they should not be imposed blindly, but programmer judgement should always take precedence. I find it very frustrating that people making a fuss about "following PEP 8" seem completely blind to the whole of the section "A Foolish Consistency is the Hobgoblin of Little Minds" (one part of which explicitly says that the PEP does not justify adding or changing code just to follow guidelines that were created after the code was written), and the *many* places where the PEP offers two (or sometimes even more) alternatives, without preferring one over the other. As I said, I'm -1 on this proposal. Paul PS If you are really committed to an alternative naming convention, you can always write a module that adds all of the aliases you might want. That way, you can follow your own preferences without imposing them on everyone else...
I'm also -1 on churning the stdlib in search of a global consistency that PEP 8 itself disavows, but this particular argument against it doesn't make sense: On Thu, Nov 11, 2021 at 9:14 AM Paul Moore <p.f.moore@gmail.com> wrote:
Any type can be called to construct an instance of that type. If I define a class Foo, I create an instance of Foo by calling `Foo(...)`. `list` and `str` are no different; I can create an instance of the type by calling it. This doesn't mean they are "both a type and a function" in some unusual way, it just means that we always call types in order to construct instances of them. Carl
On Thu, 11 Nov 2021 at 17:13, Carl Meyer <carl@oddbird.net> wrote:
I understand that. However, PEP 8 states "Names that are visible to the user as public parts of the API should follow conventions that reflect *usage* rather than *implementation*." (My emphasis) I quoted this, but you cut that part of my post. My point here is that how you interpret "usage" is far from clear - I'm sure that a lot of people would teach str(...) as a function that creates a string representation of an object, deferring the detail that it's actually a type, and you can call a type to create objects of that type until later. So would a newcomer necessarily know (or even need to know) that str is a type, not a function? There's also the case of changing implementation between a class and a factory function - surely that should not require a compatibility-breaking name change? The key is that it's fairly easy to argue "reasonable doubt" here - PEP 8 is intended to be applied with a certain level of judgement, not as a set of absolute rules. But yes, I didn't make my point particularly clearly, I apologise. Paul
On 2021-11-11 09:33, Paul Moore wrote:
I'm not the one who previously replied to your earlier post, but I still don't really understand what the relevance of this is. EVERY class can be used like a function (barring perhaps a few oddities like None). So the fact that you see a name used like `str(this)` or `list(that)` or `some_name(a, b, c)` doesn't tell you anything about "usage". That syntax is completely consistent with usage as a class and as a function. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Thu, 11 Nov 2021 at 22:22, Brendan Barnwell <brenbarn@brenbarn.net> wrote:
Chris Angelico made the point far better than I've managed to, and in any case the thread is basically finished at this point, so I won't say anything more other than to quote Chris and say "this is what I was trying to say":
Paul
I think what Paul is referring to is that according to PEP 8: - functions: Function names should be lowercase, with words separated by underscores as necessary to improve readability. - types: Class names should normally use the CapWords convention. And, of course: - Names that are visible to the user as public parts of the API should follow conventions that reflect usage rather than implementation. So, given those three items, should `str` be `str` because it is used often as a function, or should it be `Str` because it is often subclassed? -- ~Ethan~
On Thu, Nov 11, 2021 at 9:33 PM Ethan Furman <ethan@stoneleaf.us> wrote:
I understand why this idea got shut down faster than centrifuge-launched satellite, but I enjoyed reading the resulting thread and I learned a lot. Especially this idea which I have previously missed: - Names that are visible to the user as public parts of the API should follow conventions that reflect usage rather than implementation. Is there a standard idiom-- perhaps using a type-hint-- to signal to the IDE/linter that my user-defined class is intended to be used as a function/factory, and not as a type (even though it is in fact a type)? Unaware of the "reflected usage" guidelines, I have done this in the past: class _Spam: ... # todo def spam(*args): return _Spam(*args) I haven't done this often; usually it hasn't made much sense to bury the implementation into a private class like this. Most often it has been because I don't want to commit a class interface as the long term API; want to leave room to change my mind later. Seems like if there were a standard idiom for telling the linter "this class is really just kind of a factory, don't complain about the lowercase", it might be kind of nice. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On Thu, Nov 11, 2021 at 10:06:45PM -0500, Ricky Teachey wrote:
Not really. I don't think there is even a standard idiom for the human reader to decide whether something is used as a "function" or a "class". It is subjective, based on usage and convention. As others have pointed out, many functions in Python can be considered as class constructor: ord(number) # constructs a string even traditional functional programming, er, functions like map: map(func, iterable) # returns a map object I think that the best I can come up with is that if your class has a constructor that attaches input arguments (with or without additional processing) as attributes, then it might be considered "class-like". Something like this: class C: def __new__(cls, arg): obj = super().__new__(cls) obj.attribute = arg return obj or equivalently: class C: def __init__(self, arg): self.attribute = arg But if it looks like more this: class C: def __new__(cls, arg): obj = _convert(arg) # possibly call a dunder method? return obj then it is probably "function-like". But honestly, it's subjective. Some things feel like a transformation, conversion or cast using a function, and some things feel like constructing an object, even if it is impossible to find a hard, objective, bullet-proof distinction.
Any decent linter should have a switch to turn off a specific check for a line of code. For example, in flake8 the error code for class names is N801, so you should be able to disable that using: # noqa: N801 https://flake8.pycqa.org/en/3.1.1/user/ignoring-errors.html https://github.com/PyCQA/pep8-naming -- Steve
On 2021-11-12 at 14:43:07 +1100, Steven D'Aprano <steve@pearwood.info> wrote:
Isn't that why we like duck typing? I don't care what something is, I just care what it does. So when I call zip(x, y) and get an iterable, what's the difference (to me, as the user) whether zip is a class or a function or some arbitraru callable, let alone what the implementation of the resulting iterable is? If I want Java, I know where to find it.
On Fri, 12 Nov 2021 at 03:46, Steven D'Aprano <steve@pearwood.info> wrote:
Precisely. And that's why automated tools like flake8 can't reliably enforce rules like this, because they can't determine intent. On Fri, 12 Nov 2021 at 00:30, Brendan Barnwell <brenbarn@brenbarn.net> wrote:
Very much this. Tools like flake8 aren't bad in themselves (they catch when I make dumb typos in my code, and I'm grateful for that) but treating them as if they had the final say on what is acceptable code is vey bad (there's a reason linters support "#fmt: off"). Unfortunately, this usually (in my experience) comes about through a "slippery slope" of people saying that mandating a linter will stop endless debates over style preferences, as we'll just be able to say "did the linter pass?" and move on. This of course ignores the fact that (again, in my experience) far *more* time is wasted complaining about linter rules than was ever lost over arguments about style :-( Paul PS Thanks to Ethan for clarifying my posting much better than I managed to :-)
On Fri, Nov 12, 2021 at 2:22 AM Matt del Valle <matthewgdv@gmail.com> wrote:
To clarify: Your pre-commit hook is what is mandating this. Not Python. If your organization has this requirement, then it is your organization's decision how to do things. I suspect that the vast majority of Python programmers do not have pre-commit hooks that run flake8 (though I don't have stats). PEP 8 is incredibly important precisely because your organization has made it so.
Not all of those have been the same at all times. For instance, range is a function in Python 2, but a class in 3. Should it be renamed? And I'll have to get someone else to confirm, but I believe that str, int, etc were functions for a lot of Python's history.
A lot of people want this (or the similar concept "isinstance(x, collections.namedtuple)"), and it doesn't work because namedtuple is a category of classes, rather than a class itself. But suppose that were to change in the future - if namedtuple becomes a superclass or metaclass, and is then a type instead of a function. Does it immediately have to be renamed NamedTuple, or should it remain as it is? The churn has no value.
Understood. In this case, it sounds like you ARE deprecating them, but without a specific removal date.
Yes, it IS an argument against any change, and the question is: how strong are the arguments in favour of the change? With new syntax or APIs, the advantage is the increased expressiveness; with this, it's the exact same thing, but spelled differently. So your choices are (a) the thing that works all the way back to Python 2.x and will continue to work in the future; or (b) the otherwise-identical thing that only works from version X onwards. There's no incentive to move, so people won't move, so the bulk of code out there will continue to use the existing names.
This isn't about accidental shadowing - it's deliberate shadowing.
I'd also say that subclassing list as `List` is probably just bad style that should be discouraged anyway, because presumably if you're subclassing list you're doing it to extend it in some way, and you should pick a more descriptive name like `ChainableList` (if you're implementing a list where inplace methods return self and allow chaining, for example), rather than just `List`
Maybe. Sometimes, you really just want a perfectly ordinary list, but instrumented in some way. Who knows. In any case, having two names for the same thing would make this very confusing.
I think what it boils down to for me is this:
If we went to a disconnected alternate universe where python had never been invented and introduced it today, in 2021, would we introduce it with a uniform naming convention, or the historical backwards-supporting mishmash of casing we've ended up with? Since I think the answer is pretty clear, I'm strongly in favor of making this minimally-invasive change that at least works towards uniform casing, even if that dizzying utopia is far beyond the horizon. Our grandchildren might thank us :p
That's an unknowable, because things change. If str can change from being a function to a type early in Python 2.x, and range can change from being a function to a type in 3.0, then what next? Will compile become a type some day? Or chr? Would an equally hypothetical "Python invented in 2032" need to rename another bunch of things?
I do concede that some awkward shadowing edge-cases are the strongest argument against this proposal. I personally don't think they're that strong of an argument compared to the eventual payoff, but that's just my subjective opinion.
They're not the strongest argument. The strongest argument is churn - lots and lots of changes for zero benefit. The distinction between "this is a type" and "this is a function" is often relatively insignificant. The crux of your proposal is that it should be more significant, and that the fundamental APIs of various core Python callables should reflect this distinction. This is a lot of churn and only a philosophical advantage, not a practical one. This isn't the first time someone has had false expectations about PEP 8 and the standard library. I'm seriously wondering if flake8 does more harm than good by being so strict. ChrisA
Okay, so from the replies so far it looks like this is very quickly going into the 'never gonna happen' dumpster, so in the interests of salvaging *something* out of it:
See, I just dislike having to settle for 'it's what we've got'. With these two modules in particular, a lot of the arguments that have been made so far either don't apply or are not as strong (such as the confusion of having builtins.list, builtins.List and typing.List). Additionally, these are significantly more ancillary portions of the stdlib than the builtins, much less likely to cause as severe of a disruption (I personally don't believe a backward-compatible change like this which only adds aliases would be as disruptive as many people claim, but concede that that's subjective), and much less likely to have the implementation change so drastically as to want to change out types for factory functions or vice-versa. So perhaps we could narrow the scope of this down to just adding snake_case aliases to the logging and unittest modules (and any other places in the stdlib where camelCase names still exist), and leave the lowercase names alone. I'm realizing that I actually just plain forgot to list converting camelCase modules to snake_case as one of the bullet-points in the original post. That was an oversight, I did fully intend that to also be in scope. I'm very much of the philosophy of not letting perfect be the enemy of good (and yes, I know that my 'good' might be your 'evil', that's life), so if we can at least get modules like logging and unittest offering snake_case aliases for all their names I would call that at least a moderate win. With that out of the way, I'll just address a few other points: Yes, it IS an argument against any change, and the question is: how
I think saying there's no advantage is definitely a bit uncharitable. There's a gain in consistency and clarity, less surprises, and a lower cognitive load (that's the entire point of casing conventions, just by looking at a name you can glean some information about what sort of thing it references). And I think you're 100% wrong about people not switching to new names over time, especially if linters started flagging the old names as warnings and the 'best-practices' recommendation became using the new names. Someone mentioned a 20-year-old codebase too large to ever refactor, because the clients would be unwilling to pay for refactors that add no functionality. This is the exception, not the rule. I doubt even 5% of applications written in python are run for 20 years, and long-lived libraries usually have at least one or two major versions with large internal refactors over timescales that large. But that's all irrelevant anyway, because I'm not proposing removing the existing names. No backwards-compatibility will be harmed, and such projects will be able to happily go on using legacy names forever if they choose. Since most code is a lot more short-lived than that we would likely be in a position where a majority of code being actively *run* in the wild would use the new names within 15-20 years, and the vast majority of *new code* being written would be using them. They're not the strongest argument. The strongest argument is churn -
lots and lots of changes for zero benefit.
I must not be understanding what you mean by churn in this context, because to me this seems quite minor in terms of changes. From the implementation side: - A one-time addition of aliases to the stdlib - If something is deprecated/removed/renamed in the future it's alias would also have to be removed - That's it, there wouldn't be any maintenance needed beyond that, new additions would just use pep8-compliant casing From the usage side: - When writing new code, prefer the new names. You'll even be helpfully nudged along by your IDE - *If* you choose to do so, optionally do a refactor pass to change existing names in your current projects/codebases. If you choose not to, just relax your linter's legacy-name check so it doesn't bother you. The last bullet-point is the big one where I can see the argument that there would be a lot of churn. But the key thing is it's fully optional. As long as *new code* tends to use the aliases more than 50% of the time, we'll be trending in the right direction. That's an unknowable, because things change. If str can change from
But changing a type to a function and vice-versa *is* a backwards-incompatible change. If someone has written code that subclasses `collections.deque` and a new python version converts `collections.deque` into a factory function, that code will, with certainty, break when that person upgrades their interpreter to the new version. I see no problem changing a name as part of an already-breaking change. In fact, it's probably safer, with less risk of subtle bugs. The distinction between "this is a type" and "this is a function" is
As mentioned above, I don't think the distinction between functions and types is anywhere near as minor as you're suggesting. There's also the fact that reliably being able to tell if something is a type based on its casing immediately tells you useful implementation details you may be able to use, without needing to go read the documentation. For example, when I look at `builtins.range` it's basically a black box unless I dig deeper. Since I can't rely on it being a type or a function from its name alone I don't know if it will return a `range` object, or a generic generator, or something else. If it were called `builtins.Range`, I'd immediately know that I could, for example, check if something is a `Range` object with `isinstance()`. I also instantly know that I can potentially subclass it if I want to extend its functionality. That alone has value. It's not just a purely philosophical advantage, as you're suggesting. But at this stage I'm not so naive as to think this is a battle I can win, so could we refocus the discussion on the new scope which is: - Add pep8-compliant aliases for camelCased public-facing names in the stdlib (such as logging and unittest) in a similar manner as was done with threading Cheers everyone :) On Thu, Nov 11, 2021 at 5:51 PM Chris Angelico <rosuav@gmail.com> wrote:
On 2021-11-11 09:48, Chris Angelico wrote:
I think this is a big part of the problem. There are various tools out there (flake8 being one of them) that purport to "improve" or "fix" your code (or warn you to do it yourself), and various companies and organizations that adopt policies tied to those tools (e.g., "your pull request must pass this PEP 8 linter to be accepted"). It's a big problem. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
I know this suggestion is withdrawn and the thread all but finished, but for completion, I'd like to answer one of Chris' questions: On Fri, Nov 12, 2021 at 04:48:58AM +1100, Chris Angelico wrote:
I'll have to get someone else to confirm, but I believe that str, int, etc were functions for a lot of Python's history.
I don't know if the first eleven years counts as "a lot" of Python's history (it's about 1/3rd of Python's existence at this point), but in Python 1.x and some of 2.x, str, int, float, list etc were all actual functions and couldn't be subclassed: [steve@ando ~]$ python1.5 Python 1.5.2 (#1, Aug 27 2012, 09:09:18) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> >>> class MyString(str): pass ... Traceback (innermost last): File "<stdin>", line 1, in ? TypeError: base is not a class object >>> >>> type(str) <type 'builtin_function_or_method'> There were two distinct object heirarchies. Builtin *types* str, int ... were different kinds of objects to the classes and instances you created with the `class` keyword. >>> type(type('a')) <type 'type'> >>> >>> class C: pass ... >>> type(C) <type 'class'> It wasn't until Python 2.2 that builtin types and classes were unified, becoming the same thing; the functions str, int, etc became classes; and every object in Python was consolidated into a single heirarchy with `object` as the root. https://www.python.org/download/releases/2.2.3/descrintro/ -- Steve
On Fri, Nov 12, 2021 at 11:47 AM Steven D'Aprano <steve@pearwood.info> wrote:
Thanks, that's what I was thinking of. Most notably, we have several independent occasions when a builtin function was replaced with a class that could be used in all the same ways: Python 2.2 for str/int etc, Python 3.0 for range/map/zip, and I can't recall others right now, but there probably have been. BTW, when I said "someone else" there, I was thinking of you and your history lessons, so, thank you :) ChrisA
The cost of having two ways to name things for the indefinite future is too high. Not only would you have to maintain it in the various Python implementations, you'd have to explain why code uses "str" or "Str", or both. The costs of migration are also too high. I personally work on a 20 year old proprietary python code base that would never be updated for a change like this. Unless you plan on breaking it in another 20 years (and I do expect it to still be running then), the old names can never go away. Maybe if there was an automated tool, the migration to the new names would be possible. You'd need a tool that's smarter than 2to3 in order to update code like: "str = 'foo'". But even if a perfect tool existed, it would take man-months and many tens of thousands of dollars to test such a large code base. My clients are understandably unwilling to do that for no functional gain. So, like many things in Python[0], the various naming conventions across modules and time is just something we're going to have to live with. Eric [0]: One tiny example of many: I wish I hadn't added dataclasses.asdict and .astuple, but now I've got to live with it. On 11/11/2021 8:41 AM, Matt del Valle wrote:
I like this idea and disappointed it always gets a negative reaction. One of my biggest peeves is this: import datetime # or from datetime import datetime Which is often confusing... is that the datetime module or the class someone chose at random in this module? A minor thorn that… just doesn't go away. On 2021-11-11 07:38, Eric V. Smith wrote:
My current work is on a ~15 year old code base. Had to do a number of upgrades over the years. 2.x to 3.x was the big one. Luckily there was not a lot of text encoding work so porting was straightforward, and the project was improved for the effort. Despite the failures and grumbling there are success stories as well. A lot of folks are understandably hesitant at repeating the 2 vs 3 divide. But I think some learned the wrong lesson from that experience. The lesson wasn't that we shouldn't improve anything, but that we shouldn't change anything *fundamental.* Fundamental improvements generally can't be automated, they sometimes have to be rebuilt from the ground up. I agree that's a no-go. But this thread is about a rename with aliases for compatibility. Recently we brought the same project from the ~3.5 era to 3.8 idioms using the tool pyupgrade. Have you tried it? Made short work of moving forward. Project is now more readable, using better language features. It took a few hours from an existing maintenance budget—not tens of thousands of dollars. Not only that, (combined with other refactoring) the code is more fun to work on now. Yes, you read that right, enjoyment has increased due to improved readability, appearance, and quality. No, we couldn't afford to rewrite it from the ground up. But, running a tool to fix the case of a few confusing names is a small win for a small cost. I would like to continue the process. +1 for stdlib, not including typing scope creep, -Mike
Neil Girdhar wrote:
The proposal to change list and str is way too ambitious.
Maybe the uppercase versions of dict, list and tuple in the typing module could be turned into direct aliases of the built-in types? With Python 3.9, there is no real distinction anyway between builtins.list and typing.List, so might as well make the latter an alias of the former. That way, people can continue to use the lowercase versions or switch to the uppercase ones. You could then even go one step further and add uppercase aliases for str, int, float, bool to the typing module. Thomas
On 11/11/2021 14.41, Matt del Valle wrote:
Adding new APIs or replacing existing APIs for PEP 8 style-compliance would be a violation of PEP 8. The PEP 8 document states that consistency, readability, and backwards compatibility are more important than naming and style conventions. By **not** applying PEP 8 style to existing code, CPython stays PEP 8 compliant. https://www.python.org/dev/peps/pep-0008/#a-foolish-consistency-is-the-hobgo...
I'm a -1 on this proposal, as I don't see any way of doing it that wouldn't cause a huge amount of disruption. Yes, the situation — especially with regard to unittest and logging — is far from ideal. But, it's what we've got.However I'm -100 on doing something like this while there already exist a large number of camelcase typing-module aliases for classes found elsewhere in the stdlib. It's already confusing enough to have `builtins.list` and `typing.List` be different objects. Under this proposal, we'd have `builtins.list`, `builtins.List`, and `typing.List`. Two of these objects would have the same identity, but it wouldn't be the two that are spelt the same. To my mind, this would be appalling.Yes, the aliases in the typing module are all deprecated, but they're still very widely used.Best, Alex -------- Original message --------From: Matt del Valle <matthewgdv@gmail.com> Date: 11/11/2021 13:44 (GMT+00:00) To: Python-Ideas <python-ideas@python.org> Subject: [Python-ideas] Adding pep8-casing-compliant aliases for the entire stdlib So I was reading the docs for the `threading` module and I stumbled upon this little note: Note: In the Python 2.x series, this module contained camelCase names for some methods and functions. These are deprecated as of Python 3.10, but they are still supported for compatibility with Python 2.5 and lower.And it got me thinking.Given that there is some precedent, would it be feasible to make a concerted effort to add aliases across the board for all public-facing stdlib types and functions that don't follow pep8-recommended casing?I realize that large chunks of the stdlib predates pep8 and therefore use various non-uniform conventions. For example, the logging module is fully camelCased, and many core types like `str` and `list` don't use PascalCase as pep8 recommends. The `collections` module is a veritable mosaic of casing conventions, with some types like `deque` and `namedtuple` being fully lowercased while others like `Counter` and `ChainMap` are PascalCased.My motivation for this twofold:1) I'll confess that this has just always been a wart that has bothered me way more than it has any right to. I just hate it. Somewhere deep inside my lizard-brain it makes me unhappy to have disparate naming conventions in my code. I realize this isn't a good reason in and of itself but I wonder if this might not be the case for others as well. While I've come to accept it because that's just how it is, maybe it doesn't have to be this way?2) It's always been an extra thing to explain when teaching python to someone. I always try to cover pep8 very early to discourage people I'm training from internalizing bad habits, and it means you have to explain that the very standard library itself contains style violations that would get flagged in most modern code reviews, and that they just have to keep in mind that despite the fact that the core language does it, they should not.So the scope of my suggestion is as follows:- lowercase types become PascalCase (e.g., `str` -> `Str`, `collections.defaultdict` -> `collections.DefaultDict`)- lowercase attributes/functions/methods become snake_case (no changes for names that only contain a single word, so `str.lower()` would be unaffected, but `str.removeprefix()` would get the alias `str.remove_prefix()`)- pep8 and the python docs are updated to state that the pep8-compliant forms of stdlib names should be strongly preferred over the legacy names, and that IDEs and linters should include (configurable?) weak warnings to discourage the use of legacy-cased stdlib names- `help()` would be special-cased for builtin types to no longer display any current non-pep8-compliant names, and the python docs would also no longer show them, instead only making a note at the top of the page as with the `threading` module.Given the horrors of the python 2.7 schism I don't think there's any rush to officially deprecate or remove the current non-pep8 names at all. I think that's the sort of thing that can happily and fully be kicked down the road. If we add aliases and they see widespread adoption to the point where the non-pep8 forms are barely ever even seen out in the wild then maybe in 10 or 20 years time when the steering council is deliberating on a new major python version they can consider rolling the removal of legacy badly-cased names into it. And if not then no big deal.So yeah, thoughts?
One thought: No.
I was still typing out my last reply when you wrote this Guido. With that I think I will officially retract the entire suggestion. Though I'm disappointed because casing-consistency is one of those things I care about far more than I probably should (I couldn't explain the why, I just do), I respect your opinion massively as I'm sure most of our community does, so I don't see any benefit in continuing the discussion if you're soundly against it. Thanks to everyone who chimed in with their opinions, have a good rest of your day :) On Thu, Nov 11, 2021 at 7:53 PM Guido van Rossum <guido@python.org> wrote:
I'll simply say that while I am disappointed that there are so many frictions here for whatever reason, I *strongly* appreciate the work done to update threading, I get a deep sense of satisfaction using *correct* naming conventions whenever using this library, and am glad that it got through somehow On Thu, Nov 11, 2021 at 12:01 PM Matt del Valle <matthewgdv@gmail.com> wrote:
On 11/13/21 9:59 AM, Stephen J. Turnbull wrote:
Or even import nonPEP8module nonPEP8modue.pep8name = nonPEP8module.nonpep8name to monkeypatch the module to have the pep8name in it. As long as you import this module before you try it, you can then even do from nonPEP8module import pep8nane -- Richard Damon
You're not alone—it bothers me too! I try to use them as an example of why I shouldn't obsess over all the details (a certain "hobgoblin" quote always comes to mind), but I would *always* use the more consistent version if it were to exist... On Thu, Nov 11, 2021 at 8:42 AM Matt del Valle <matthewgdv@gmail.com> wrote:
On Fri, Nov 12, 2021 at 12:41 AM Matt del Valle <matthewgdv@gmail.com> wrote:
2) It's always been an extra thing to explain when teaching python to someone. I always try to cover pep8 very early to discourage people I'm training from internalizing bad habits, and it means you have to explain that the very standard library itself contains style violations that would get flagged in most modern code reviews, and that they just have to keep in mind that despite the fact that the core language does it, they should not.
ISTM that this indicates that you're putting too much focus on PEP 8 too early. At no time does the document ever state that all Python code ever written must comply with it. New Python programmers should not feel like they're being forced into a naming convention.
So the scope of my suggestion is as follows:
- lowercase types become PascalCase (e.g., `str` -> `Str`, `collections.defaultdict` -> `collections.DefaultDict`)
Pop quiz: Which of these are types and which are functions (or something else)? bool, classmethod, divmod, enumerate, globals, map, property, sorted, super, zip collections.deque, collections.namedtuple Does it even matter? Especially: does it matter enough to force a name change if a function is replaced with a type, or vice versa?
- lowercase attributes/functions/methods become snake_case (no changes for names that only contain a single word, so `str.lower()` would be unaffected, but `str.removeprefix()` would get the alias `str.remove_prefix()`)
Are you aware that removeprefix is very new, and that the alternate name remove_prefix was rejected? https://www.python.org/dev/peps/pep-0616/#alternative-method-names
Given the horrors of the python 2.7 schism I don't think there's any rush to officially deprecate or remove the current non-pep8 names at all. I think that's the sort of thing that can happily and fully be kicked down the road.
Wait, are you deprecating them or not? I'm confused. Earlier you were saying that you clearly wanted to stop people from using the old names, but now you're saying there's no rush to deprecate them.
If we add aliases and they see widespread adoption to the point where the non-pep8 forms are barely ever even seen out in the wild then maybe in 10 or 20 years time when the steering council is deliberating on a new major python version they can consider rolling the removal of legacy badly-cased names into it. And if not then no big deal.
That will never happen. It's still possible today to find code written for older 2.x versions, including some quite popular answers on places like Stack Overflow. Suppose that these aliases were added in Python 3.11. Anyone who wants to write code compatible with 3.10 would want to continue using the existing names, and since the existing names would keep on being supported for the foreseeable future, there would be little incentive to change until several versions have passed. At that point, both names might start being used in parallel (we're talking probably 2025 or thereabouts, although it might start sooner if people are also using syntactic features from 3.11+), but it would take a VERY long time for adoption to the level that the older names are "barely ever even seen". Probably never.
So yeah, thoughts?
Absolutely no value in adding aliases for everything, especially things that can be shadowed. It's not hugely common, but suppose that you deliberately shadow the name "list" in your project - now the List alias has become disconnected from it, unless you explicitly shadow that one as well. Conversely, a much more common practice is to actually use the capitalized version as a variant: class List(list): ... This would now be shadowing just one, but not the other, of the built-ins. Confusion would abound. In closing, I'd just like to highlight one very important section of PEP 8: https://www.python.org/dev/peps/pep-0008/#a-foolish-consistency-is-the-hobgo... When a style guide becomes a boat anchor, it's not doing its job. ChrisA
That's fair enough for people learning python as a hobby or in a context that's a bit more casual than agile. I generally find myself training junior/graduate people with OOP backgrounds other than Python (Java, C#, etc.) for my data engineering team and pep8-compliance is quite important, because they won't even be able to successfully make commits unless they can pass the flake8 pre-commit hook, let alone get as far as a merge review. So I'd say that pep8 is quite important from the get-go. Pop quiz: Which of these are types and which are functions (or something
I'd argue this is actually a point *in favor* of the proposal rather than against. In going through your list I actually discovered lots of things I found extremely surprising (collections.deque is a type as I expected whereas collections.namedtuple is actually a factory function). If pep8 had been introduced at the dawn of (python-)time and the stdlib had been designed with pep8-compliance in mind, there would be no surprise at all on the day that I think to myself: 'Hmm, I want to subclass namedtuple, let's try:' class MyNamedTuple(collections.namedtuple): ... and discover that it doesn't work. Are you aware that removeprefix is very new, and that the alternate
name remove_prefix was rejected?
I was aware of that, yes. The rationale was precisely to introduce it as `str.removeprefix` for consistency with alllowercase (<- which all other considerations aside can be very unreadable sometimes, that's, I assume, why pep8 prefers snake_case) which is itself in contravention of python's official style guide. This is irrelevant in the event that this proposal is accepted. Wait, are you deprecating them or not? I'm confused. Earlier you were
saying that you clearly wanted to stop people from using the old names, but now you're saying there's no rush to deprecate them.
This is a semantic misunderstanding, I should have been clearer. In my head, 'officially deprecating' something means saying that it will *definitely* be removed in the future, and optionally specifying the precise date at which it will be removed. Since my suggestion doesn't necessarily involve ever actually removing the current names from the language (only potentially, if a future steering council decides to as part of a future proposal), I didn't consider this to be deprecating them, exactly. Hopefully that's cleared up. Sorry for the confusing wording. Suppose that these aliases were added in Python 3.11. Anyone who wants
This is an argument that can be made for any change ever made to python, and consequently I think it's an extremely weak argument. Why change anything if people won't be confident using it for at least 5 years because of compatibility concerns with current language versions? Well, as it turns out time passes and eventually that distant 5-year mark is in the past. I just checked and... wow yeah f-strings were introduced in 2016! Feels like only yesterday :) Absolutely no value in adding aliases for everything, especially
I think this is a fair point, but the same can be said of people accidentally shadowing any number of other builtins (it's probably most common with builtins like `type` or `max`). Fortunately every linter/IDE I've used has had warnings for this so it should be something that happens rarely, if ever. I'd also say that subclassing list as `List` is probably just bad style that should be discouraged anyway, because presumably if you're subclassing list you're doing it to extend it in some way, and you should pick a more descriptive name like `ChainableList` (if you're implementing a list where inplace methods return self and allow chaining, for example), rather than just `List` I'd argue that a far more common usage pattern is to assign an instance to the snake_case version of it's class name, for example: email_service = EmailService(...) or for the example in question: list = List() I think this is an entirely reasonable usage form for a hypothetical future codebase that disallows legacy-cased names and enforces it with flake8. But currently this can't be done. I think what it boils down to for me is this: If we went to a disconnected alternate universe where python had never been invented and introduced it today, in 2021, would we introduce it with a uniform naming convention, or the historical backwards-supporting mishmash of casing we've ended up with? Since I think the answer is pretty clear, I'm strongly in favor of making this minimally-invasive change that at least works towards uniform casing, even if that dizzying utopia is far beyond the horizon. Our grandchildren might thank us :p I do concede that some awkward shadowing edge-cases are the strongest argument against this proposal. I personally don't think they're that strong of an argument compared to the eventual payoff, but that's just my subjective opinion. On Thu, Nov 11, 2021 at 2:11 PM Chris Angelico <rosuav@gmail.com> wrote:
Let's just say up front that I'm a strong -1 on this proposal, as I think it is needless churn, and while it may be *technically* backward compatible, in reality it will be immensely disruptive. There's one particular point I want to pick up on, though. On Thu, 11 Nov 2021 at 15:25, Matt del Valle <matthewgdv@gmail.com> wrote:
If we went to a disconnected alternate universe where python had never been invented and introduced it today, in 2021, would we introduce it with a uniform naming convention, or the historical backwards-supporting mishmash of casing we've ended up with? Since I think the answer is pretty clear, I'm strongly in favor of making this minimally-invasive change that at least works towards uniform casing, even if that dizzying utopia is far beyond the horizon. Our grandchildren might thank us :p
Certainly, there's a lot of inconsistency that can only be justified as historical baggage. There have been a number of proposals to "tidy up" such cases, but even when focused on specific instances, the general conclusion has been that this would be too disruptive. But let's put that aside and address the broader question here. Yes, if we'd been designing everything now, we quite probably would have adopted a more consistent approach. But I think it's foolish to assume that whatever convention we defined for names would be strictly based on the type of the value. After all, even if you adopt a no-compromises stance on PEP 8 (a stance that the PEP itself rejects, by the way!) the first part of the "Naming Conventions" section says """ Names that are visible to the user as public parts of the API should follow conventions that reflect usage rather than implementation. """ To examine some specific cases, lists are a type, but list(...) is a function for constructing lists. The function-style usage is far more common than the use of list as a type name (possibly depending on how much of a static typing advocate you are...). So "list" should be lower case by that logic, and therefore according to PEP 8. And str() is a function for getting the string representation of an object as well as being a type - so should it be "str" or "Str"? That's at best a judgement call (usage is probably more evenly divided in this case), but PEP 8 supports both choices. Or to put it another way, "uniform" casing is a myth, if you read PEP 8 properly. What you actually seem to be arguing for is a renaming based on a hypothetical version of PEP 8 that is far stricter than the actual document, and which doesn't take into account the messiness of real-world APIs and applications. That's a very common, and in my opinion misguided, stance. For me, one of the best things about PEP 8 is its repeated assertions that the "rules" it defines are only guidelines and that they should not be imposed blindly, but programmer judgement should always take precedence. I find it very frustrating that people making a fuss about "following PEP 8" seem completely blind to the whole of the section "A Foolish Consistency is the Hobgoblin of Little Minds" (one part of which explicitly says that the PEP does not justify adding or changing code just to follow guidelines that were created after the code was written), and the *many* places where the PEP offers two (or sometimes even more) alternatives, without preferring one over the other. As I said, I'm -1 on this proposal. Paul PS If you are really committed to an alternative naming convention, you can always write a module that adds all of the aliases you might want. That way, you can follow your own preferences without imposing them on everyone else...
I'm also -1 on churning the stdlib in search of a global consistency that PEP 8 itself disavows, but this particular argument against it doesn't make sense: On Thu, Nov 11, 2021 at 9:14 AM Paul Moore <p.f.moore@gmail.com> wrote:
Any type can be called to construct an instance of that type. If I define a class Foo, I create an instance of Foo by calling `Foo(...)`. `list` and `str` are no different; I can create an instance of the type by calling it. This doesn't mean they are "both a type and a function" in some unusual way, it just means that we always call types in order to construct instances of them. Carl
On Thu, 11 Nov 2021 at 17:13, Carl Meyer <carl@oddbird.net> wrote:
I understand that. However, PEP 8 states "Names that are visible to the user as public parts of the API should follow conventions that reflect *usage* rather than *implementation*." (My emphasis) I quoted this, but you cut that part of my post. My point here is that how you interpret "usage" is far from clear - I'm sure that a lot of people would teach str(...) as a function that creates a string representation of an object, deferring the detail that it's actually a type, and you can call a type to create objects of that type until later. So would a newcomer necessarily know (or even need to know) that str is a type, not a function? There's also the case of changing implementation between a class and a factory function - surely that should not require a compatibility-breaking name change? The key is that it's fairly easy to argue "reasonable doubt" here - PEP 8 is intended to be applied with a certain level of judgement, not as a set of absolute rules. But yes, I didn't make my point particularly clearly, I apologise. Paul
On 2021-11-11 09:33, Paul Moore wrote:
I'm not the one who previously replied to your earlier post, but I still don't really understand what the relevance of this is. EVERY class can be used like a function (barring perhaps a few oddities like None). So the fact that you see a name used like `str(this)` or `list(that)` or `some_name(a, b, c)` doesn't tell you anything about "usage". That syntax is completely consistent with usage as a class and as a function. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Thu, 11 Nov 2021 at 22:22, Brendan Barnwell <brenbarn@brenbarn.net> wrote:
Chris Angelico made the point far better than I've managed to, and in any case the thread is basically finished at this point, so I won't say anything more other than to quote Chris and say "this is what I was trying to say":
Paul
I think what Paul is referring to is that according to PEP 8: - functions: Function names should be lowercase, with words separated by underscores as necessary to improve readability. - types: Class names should normally use the CapWords convention. And, of course: - Names that are visible to the user as public parts of the API should follow conventions that reflect usage rather than implementation. So, given those three items, should `str` be `str` because it is used often as a function, or should it be `Str` because it is often subclassed? -- ~Ethan~
On Thu, Nov 11, 2021 at 9:33 PM Ethan Furman <ethan@stoneleaf.us> wrote:
I understand why this idea got shut down faster than centrifuge-launched satellite, but I enjoyed reading the resulting thread and I learned a lot. Especially this idea which I have previously missed: - Names that are visible to the user as public parts of the API should follow conventions that reflect usage rather than implementation. Is there a standard idiom-- perhaps using a type-hint-- to signal to the IDE/linter that my user-defined class is intended to be used as a function/factory, and not as a type (even though it is in fact a type)? Unaware of the "reflected usage" guidelines, I have done this in the past: class _Spam: ... # todo def spam(*args): return _Spam(*args) I haven't done this often; usually it hasn't made much sense to bury the implementation into a private class like this. Most often it has been because I don't want to commit a class interface as the long term API; want to leave room to change my mind later. Seems like if there were a standard idiom for telling the linter "this class is really just kind of a factory, don't complain about the lowercase", it might be kind of nice. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On Thu, Nov 11, 2021 at 10:06:45PM -0500, Ricky Teachey wrote:
Not really. I don't think there is even a standard idiom for the human reader to decide whether something is used as a "function" or a "class". It is subjective, based on usage and convention. As others have pointed out, many functions in Python can be considered as class constructor: ord(number) # constructs a string even traditional functional programming, er, functions like map: map(func, iterable) # returns a map object I think that the best I can come up with is that if your class has a constructor that attaches input arguments (with or without additional processing) as attributes, then it might be considered "class-like". Something like this: class C: def __new__(cls, arg): obj = super().__new__(cls) obj.attribute = arg return obj or equivalently: class C: def __init__(self, arg): self.attribute = arg But if it looks like more this: class C: def __new__(cls, arg): obj = _convert(arg) # possibly call a dunder method? return obj then it is probably "function-like". But honestly, it's subjective. Some things feel like a transformation, conversion or cast using a function, and some things feel like constructing an object, even if it is impossible to find a hard, objective, bullet-proof distinction.
Any decent linter should have a switch to turn off a specific check for a line of code. For example, in flake8 the error code for class names is N801, so you should be able to disable that using: # noqa: N801 https://flake8.pycqa.org/en/3.1.1/user/ignoring-errors.html https://github.com/PyCQA/pep8-naming -- Steve
On 2021-11-12 at 14:43:07 +1100, Steven D'Aprano <steve@pearwood.info> wrote:
Isn't that why we like duck typing? I don't care what something is, I just care what it does. So when I call zip(x, y) and get an iterable, what's the difference (to me, as the user) whether zip is a class or a function or some arbitraru callable, let alone what the implementation of the resulting iterable is? If I want Java, I know where to find it.
On Fri, 12 Nov 2021 at 03:46, Steven D'Aprano <steve@pearwood.info> wrote:
Precisely. And that's why automated tools like flake8 can't reliably enforce rules like this, because they can't determine intent. On Fri, 12 Nov 2021 at 00:30, Brendan Barnwell <brenbarn@brenbarn.net> wrote:
Very much this. Tools like flake8 aren't bad in themselves (they catch when I make dumb typos in my code, and I'm grateful for that) but treating them as if they had the final say on what is acceptable code is vey bad (there's a reason linters support "#fmt: off"). Unfortunately, this usually (in my experience) comes about through a "slippery slope" of people saying that mandating a linter will stop endless debates over style preferences, as we'll just be able to say "did the linter pass?" and move on. This of course ignores the fact that (again, in my experience) far *more* time is wasted complaining about linter rules than was ever lost over arguments about style :-( Paul PS Thanks to Ethan for clarifying my posting much better than I managed to :-)
On Fri, Nov 12, 2021 at 2:22 AM Matt del Valle <matthewgdv@gmail.com> wrote:
To clarify: Your pre-commit hook is what is mandating this. Not Python. If your organization has this requirement, then it is your organization's decision how to do things. I suspect that the vast majority of Python programmers do not have pre-commit hooks that run flake8 (though I don't have stats). PEP 8 is incredibly important precisely because your organization has made it so.
Not all of those have been the same at all times. For instance, range is a function in Python 2, but a class in 3. Should it be renamed? And I'll have to get someone else to confirm, but I believe that str, int, etc were functions for a lot of Python's history.
A lot of people want this (or the similar concept "isinstance(x, collections.namedtuple)"), and it doesn't work because namedtuple is a category of classes, rather than a class itself. But suppose that were to change in the future - if namedtuple becomes a superclass or metaclass, and is then a type instead of a function. Does it immediately have to be renamed NamedTuple, or should it remain as it is? The churn has no value.
Understood. In this case, it sounds like you ARE deprecating them, but without a specific removal date.
Yes, it IS an argument against any change, and the question is: how strong are the arguments in favour of the change? With new syntax or APIs, the advantage is the increased expressiveness; with this, it's the exact same thing, but spelled differently. So your choices are (a) the thing that works all the way back to Python 2.x and will continue to work in the future; or (b) the otherwise-identical thing that only works from version X onwards. There's no incentive to move, so people won't move, so the bulk of code out there will continue to use the existing names.
This isn't about accidental shadowing - it's deliberate shadowing.
I'd also say that subclassing list as `List` is probably just bad style that should be discouraged anyway, because presumably if you're subclassing list you're doing it to extend it in some way, and you should pick a more descriptive name like `ChainableList` (if you're implementing a list where inplace methods return self and allow chaining, for example), rather than just `List`
Maybe. Sometimes, you really just want a perfectly ordinary list, but instrumented in some way. Who knows. In any case, having two names for the same thing would make this very confusing.
I think what it boils down to for me is this:
If we went to a disconnected alternate universe where python had never been invented and introduced it today, in 2021, would we introduce it with a uniform naming convention, or the historical backwards-supporting mishmash of casing we've ended up with? Since I think the answer is pretty clear, I'm strongly in favor of making this minimally-invasive change that at least works towards uniform casing, even if that dizzying utopia is far beyond the horizon. Our grandchildren might thank us :p
That's an unknowable, because things change. If str can change from being a function to a type early in Python 2.x, and range can change from being a function to a type in 3.0, then what next? Will compile become a type some day? Or chr? Would an equally hypothetical "Python invented in 2032" need to rename another bunch of things?
I do concede that some awkward shadowing edge-cases are the strongest argument against this proposal. I personally don't think they're that strong of an argument compared to the eventual payoff, but that's just my subjective opinion.
They're not the strongest argument. The strongest argument is churn - lots and lots of changes for zero benefit. The distinction between "this is a type" and "this is a function" is often relatively insignificant. The crux of your proposal is that it should be more significant, and that the fundamental APIs of various core Python callables should reflect this distinction. This is a lot of churn and only a philosophical advantage, not a practical one. This isn't the first time someone has had false expectations about PEP 8 and the standard library. I'm seriously wondering if flake8 does more harm than good by being so strict. ChrisA
Okay, so from the replies so far it looks like this is very quickly going into the 'never gonna happen' dumpster, so in the interests of salvaging *something* out of it:
See, I just dislike having to settle for 'it's what we've got'. With these two modules in particular, a lot of the arguments that have been made so far either don't apply or are not as strong (such as the confusion of having builtins.list, builtins.List and typing.List). Additionally, these are significantly more ancillary portions of the stdlib than the builtins, much less likely to cause as severe of a disruption (I personally don't believe a backward-compatible change like this which only adds aliases would be as disruptive as many people claim, but concede that that's subjective), and much less likely to have the implementation change so drastically as to want to change out types for factory functions or vice-versa. So perhaps we could narrow the scope of this down to just adding snake_case aliases to the logging and unittest modules (and any other places in the stdlib where camelCase names still exist), and leave the lowercase names alone. I'm realizing that I actually just plain forgot to list converting camelCase modules to snake_case as one of the bullet-points in the original post. That was an oversight, I did fully intend that to also be in scope. I'm very much of the philosophy of not letting perfect be the enemy of good (and yes, I know that my 'good' might be your 'evil', that's life), so if we can at least get modules like logging and unittest offering snake_case aliases for all their names I would call that at least a moderate win. With that out of the way, I'll just address a few other points: Yes, it IS an argument against any change, and the question is: how
I think saying there's no advantage is definitely a bit uncharitable. There's a gain in consistency and clarity, less surprises, and a lower cognitive load (that's the entire point of casing conventions, just by looking at a name you can glean some information about what sort of thing it references). And I think you're 100% wrong about people not switching to new names over time, especially if linters started flagging the old names as warnings and the 'best-practices' recommendation became using the new names. Someone mentioned a 20-year-old codebase too large to ever refactor, because the clients would be unwilling to pay for refactors that add no functionality. This is the exception, not the rule. I doubt even 5% of applications written in python are run for 20 years, and long-lived libraries usually have at least one or two major versions with large internal refactors over timescales that large. But that's all irrelevant anyway, because I'm not proposing removing the existing names. No backwards-compatibility will be harmed, and such projects will be able to happily go on using legacy names forever if they choose. Since most code is a lot more short-lived than that we would likely be in a position where a majority of code being actively *run* in the wild would use the new names within 15-20 years, and the vast majority of *new code* being written would be using them. They're not the strongest argument. The strongest argument is churn -
lots and lots of changes for zero benefit.
I must not be understanding what you mean by churn in this context, because to me this seems quite minor in terms of changes. From the implementation side: - A one-time addition of aliases to the stdlib - If something is deprecated/removed/renamed in the future it's alias would also have to be removed - That's it, there wouldn't be any maintenance needed beyond that, new additions would just use pep8-compliant casing From the usage side: - When writing new code, prefer the new names. You'll even be helpfully nudged along by your IDE - *If* you choose to do so, optionally do a refactor pass to change existing names in your current projects/codebases. If you choose not to, just relax your linter's legacy-name check so it doesn't bother you. The last bullet-point is the big one where I can see the argument that there would be a lot of churn. But the key thing is it's fully optional. As long as *new code* tends to use the aliases more than 50% of the time, we'll be trending in the right direction. That's an unknowable, because things change. If str can change from
But changing a type to a function and vice-versa *is* a backwards-incompatible change. If someone has written code that subclasses `collections.deque` and a new python version converts `collections.deque` into a factory function, that code will, with certainty, break when that person upgrades their interpreter to the new version. I see no problem changing a name as part of an already-breaking change. In fact, it's probably safer, with less risk of subtle bugs. The distinction between "this is a type" and "this is a function" is
As mentioned above, I don't think the distinction between functions and types is anywhere near as minor as you're suggesting. There's also the fact that reliably being able to tell if something is a type based on its casing immediately tells you useful implementation details you may be able to use, without needing to go read the documentation. For example, when I look at `builtins.range` it's basically a black box unless I dig deeper. Since I can't rely on it being a type or a function from its name alone I don't know if it will return a `range` object, or a generic generator, or something else. If it were called `builtins.Range`, I'd immediately know that I could, for example, check if something is a `Range` object with `isinstance()`. I also instantly know that I can potentially subclass it if I want to extend its functionality. That alone has value. It's not just a purely philosophical advantage, as you're suggesting. But at this stage I'm not so naive as to think this is a battle I can win, so could we refocus the discussion on the new scope which is: - Add pep8-compliant aliases for camelCased public-facing names in the stdlib (such as logging and unittest) in a similar manner as was done with threading Cheers everyone :) On Thu, Nov 11, 2021 at 5:51 PM Chris Angelico <rosuav@gmail.com> wrote:
On 2021-11-11 09:48, Chris Angelico wrote:
I think this is a big part of the problem. There are various tools out there (flake8 being one of them) that purport to "improve" or "fix" your code (or warn you to do it yourself), and various companies and organizations that adopt policies tied to those tools (e.g., "your pull request must pass this PEP 8 linter to be accepted"). It's a big problem. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
I know this suggestion is withdrawn and the thread all but finished, but for completion, I'd like to answer one of Chris' questions: On Fri, Nov 12, 2021 at 04:48:58AM +1100, Chris Angelico wrote:
I'll have to get someone else to confirm, but I believe that str, int, etc were functions for a lot of Python's history.
I don't know if the first eleven years counts as "a lot" of Python's history (it's about 1/3rd of Python's existence at this point), but in Python 1.x and some of 2.x, str, int, float, list etc were all actual functions and couldn't be subclassed: [steve@ando ~]$ python1.5 Python 1.5.2 (#1, Aug 27 2012, 09:09:18) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> >>> class MyString(str): pass ... Traceback (innermost last): File "<stdin>", line 1, in ? TypeError: base is not a class object >>> >>> type(str) <type 'builtin_function_or_method'> There were two distinct object heirarchies. Builtin *types* str, int ... were different kinds of objects to the classes and instances you created with the `class` keyword. >>> type(type('a')) <type 'type'> >>> >>> class C: pass ... >>> type(C) <type 'class'> It wasn't until Python 2.2 that builtin types and classes were unified, becoming the same thing; the functions str, int, etc became classes; and every object in Python was consolidated into a single heirarchy with `object` as the root. https://www.python.org/download/releases/2.2.3/descrintro/ -- Steve
On Fri, Nov 12, 2021 at 11:47 AM Steven D'Aprano <steve@pearwood.info> wrote:
Thanks, that's what I was thinking of. Most notably, we have several independent occasions when a builtin function was replaced with a class that could be used in all the same ways: Python 2.2 for str/int etc, Python 3.0 for range/map/zip, and I can't recall others right now, but there probably have been. BTW, when I said "someone else" there, I was thinking of you and your history lessons, so, thank you :) ChrisA
The cost of having two ways to name things for the indefinite future is too high. Not only would you have to maintain it in the various Python implementations, you'd have to explain why code uses "str" or "Str", or both. The costs of migration are also too high. I personally work on a 20 year old proprietary python code base that would never be updated for a change like this. Unless you plan on breaking it in another 20 years (and I do expect it to still be running then), the old names can never go away. Maybe if there was an automated tool, the migration to the new names would be possible. You'd need a tool that's smarter than 2to3 in order to update code like: "str = 'foo'". But even if a perfect tool existed, it would take man-months and many tens of thousands of dollars to test such a large code base. My clients are understandably unwilling to do that for no functional gain. So, like many things in Python[0], the various naming conventions across modules and time is just something we're going to have to live with. Eric [0]: One tiny example of many: I wish I hadn't added dataclasses.asdict and .astuple, but now I've got to live with it. On 11/11/2021 8:41 AM, Matt del Valle wrote:
I like this idea and disappointed it always gets a negative reaction. One of my biggest peeves is this: import datetime # or from datetime import datetime Which is often confusing... is that the datetime module or the class someone chose at random in this module? A minor thorn that… just doesn't go away. On 2021-11-11 07:38, Eric V. Smith wrote:
My current work is on a ~15 year old code base. Had to do a number of upgrades over the years. 2.x to 3.x was the big one. Luckily there was not a lot of text encoding work so porting was straightforward, and the project was improved for the effort. Despite the failures and grumbling there are success stories as well. A lot of folks are understandably hesitant at repeating the 2 vs 3 divide. But I think some learned the wrong lesson from that experience. The lesson wasn't that we shouldn't improve anything, but that we shouldn't change anything *fundamental.* Fundamental improvements generally can't be automated, they sometimes have to be rebuilt from the ground up. I agree that's a no-go. But this thread is about a rename with aliases for compatibility. Recently we brought the same project from the ~3.5 era to 3.8 idioms using the tool pyupgrade. Have you tried it? Made short work of moving forward. Project is now more readable, using better language features. It took a few hours from an existing maintenance budget—not tens of thousands of dollars. Not only that, (combined with other refactoring) the code is more fun to work on now. Yes, you read that right, enjoyment has increased due to improved readability, appearance, and quality. No, we couldn't afford to rewrite it from the ground up. But, running a tool to fix the case of a few confusing names is a small win for a small cost. I would like to continue the process. +1 for stdlib, not including typing scope creep, -Mike
Neil Girdhar wrote:
The proposal to change list and str is way too ambitious.
Maybe the uppercase versions of dict, list and tuple in the typing module could be turned into direct aliases of the built-in types? With Python 3.9, there is no real distinction anyway between builtins.list and typing.List, so might as well make the latter an alias of the former. That way, people can continue to use the lowercase versions or switch to the uppercase ones. You could then even go one step further and add uppercase aliases for str, int, float, bool to the typing module. Thomas
On 11/11/2021 14.41, Matt del Valle wrote:
Adding new APIs or replacing existing APIs for PEP 8 style-compliance would be a violation of PEP 8. The PEP 8 document states that consistency, readability, and backwards compatibility are more important than naming and style conventions. By **not** applying PEP 8 style to existing code, CPython stays PEP 8 compliant. https://www.python.org/dev/peps/pep-0008/#a-foolish-consistency-is-the-hobgo...
I'm a -1 on this proposal, as I don't see any way of doing it that wouldn't cause a huge amount of disruption. Yes, the situation — especially with regard to unittest and logging — is far from ideal. But, it's what we've got.However I'm -100 on doing something like this while there already exist a large number of camelcase typing-module aliases for classes found elsewhere in the stdlib. It's already confusing enough to have `builtins.list` and `typing.List` be different objects. Under this proposal, we'd have `builtins.list`, `builtins.List`, and `typing.List`. Two of these objects would have the same identity, but it wouldn't be the two that are spelt the same. To my mind, this would be appalling.Yes, the aliases in the typing module are all deprecated, but they're still very widely used.Best, Alex -------- Original message --------From: Matt del Valle <matthewgdv@gmail.com> Date: 11/11/2021 13:44 (GMT+00:00) To: Python-Ideas <python-ideas@python.org> Subject: [Python-ideas] Adding pep8-casing-compliant aliases for the entire stdlib So I was reading the docs for the `threading` module and I stumbled upon this little note: Note: In the Python 2.x series, this module contained camelCase names for some methods and functions. These are deprecated as of Python 3.10, but they are still supported for compatibility with Python 2.5 and lower.And it got me thinking.Given that there is some precedent, would it be feasible to make a concerted effort to add aliases across the board for all public-facing stdlib types and functions that don't follow pep8-recommended casing?I realize that large chunks of the stdlib predates pep8 and therefore use various non-uniform conventions. For example, the logging module is fully camelCased, and many core types like `str` and `list` don't use PascalCase as pep8 recommends. The `collections` module is a veritable mosaic of casing conventions, with some types like `deque` and `namedtuple` being fully lowercased while others like `Counter` and `ChainMap` are PascalCased.My motivation for this twofold:1) I'll confess that this has just always been a wart that has bothered me way more than it has any right to. I just hate it. Somewhere deep inside my lizard-brain it makes me unhappy to have disparate naming conventions in my code. I realize this isn't a good reason in and of itself but I wonder if this might not be the case for others as well. While I've come to accept it because that's just how it is, maybe it doesn't have to be this way?2) It's always been an extra thing to explain when teaching python to someone. I always try to cover pep8 very early to discourage people I'm training from internalizing bad habits, and it means you have to explain that the very standard library itself contains style violations that would get flagged in most modern code reviews, and that they just have to keep in mind that despite the fact that the core language does it, they should not.So the scope of my suggestion is as follows:- lowercase types become PascalCase (e.g., `str` -> `Str`, `collections.defaultdict` -> `collections.DefaultDict`)- lowercase attributes/functions/methods become snake_case (no changes for names that only contain a single word, so `str.lower()` would be unaffected, but `str.removeprefix()` would get the alias `str.remove_prefix()`)- pep8 and the python docs are updated to state that the pep8-compliant forms of stdlib names should be strongly preferred over the legacy names, and that IDEs and linters should include (configurable?) weak warnings to discourage the use of legacy-cased stdlib names- `help()` would be special-cased for builtin types to no longer display any current non-pep8-compliant names, and the python docs would also no longer show them, instead only making a note at the top of the page as with the `threading` module.Given the horrors of the python 2.7 schism I don't think there's any rush to officially deprecate or remove the current non-pep8 names at all. I think that's the sort of thing that can happily and fully be kicked down the road. If we add aliases and they see widespread adoption to the point where the non-pep8 forms are barely ever even seen out in the wild then maybe in 10 or 20 years time when the steering council is deliberating on a new major python version they can consider rolling the removal of legacy badly-cased names into it. And if not then no big deal.So yeah, thoughts?
One thought: No.
I was still typing out my last reply when you wrote this Guido. With that I think I will officially retract the entire suggestion. Though I'm disappointed because casing-consistency is one of those things I care about far more than I probably should (I couldn't explain the why, I just do), I respect your opinion massively as I'm sure most of our community does, so I don't see any benefit in continuing the discussion if you're soundly against it. Thanks to everyone who chimed in with their opinions, have a good rest of your day :) On Thu, Nov 11, 2021 at 7:53 PM Guido van Rossum <guido@python.org> wrote:
I'll simply say that while I am disappointed that there are so many frictions here for whatever reason, I *strongly* appreciate the work done to update threading, I get a deep sense of satisfaction using *correct* naming conventions whenever using this library, and am glad that it got through somehow On Thu, Nov 11, 2021 at 12:01 PM Matt del Valle <matthewgdv@gmail.com> wrote:
On 11/13/21 9:59 AM, Stephen J. Turnbull wrote:
Or even import nonPEP8module nonPEP8modue.pep8name = nonPEP8module.nonpep8name to monkeypatch the module to have the pep8name in it. As long as you import this module before you try it, you can then even do from nonPEP8module import pep8nane -- Richard Damon
participants (21)
-
2QdxY4RzWzUUiLuE＠potatochowder.com -
Alex Waygood -
Brendan Barnwell -
Carl Meyer -
Chris Angelico -
Christian Heimes -
Eric V. Smith -
Ethan Furman -
Guido van Rossum -
Jelle Zijlstra -
Mark Mollineaux -
Matt del Valle -
Mike Miller -
Mitch -
Neil Girdhar -
Paul Moore -
Richard Damon -
Ricky Teachey -
Stephen J. Turnbull -
Steven D'Aprano -
tmkehrenberg＠gmail.com