If you feel so strongly about it maybe we need to see some credentials. What's your background? (This may sound like an odd request, but reputation matters, and right now I have no idea who you are or why I should take your opinion seriously, no matter how eloquently it is stated.)

Note that C extensions allow none of the introspection mechanisms you're proposing here (they have a `__dict__` but it is fully controlled by the author of the C code) and they thrive just fine.

I am happy with giving module authors a way to tell the system "it's okay, users can reach in to access other attributes as well". But I think module authors should *also* be given a way to tell the system "please prevent users from accessing the private parts of this API no matter how hard they try". And using 'export' seems a reasonable way to enable the latter. (To be clear I am fine if there's a flag to override this default even when 'export' is used.)

I know there are package authors out there who have this desire to restrict access (presumably because they've been burned when trying to evolve their API) and feel so strongly about it that they implement their own restrictive access controls (without resorting to writing C code).

On Sun, Mar 14, 2021 at 12:42 PM Stestagg <stestagg@gmail.com> wrote:

On Sun, 14 Mar 2021 at 18:58, Guido van Rossum <guido@python.org> wrote:
On Sun, Mar 14, 2021 at 7:11 AM Theia Vogel <theia@vgel.me> wrote:
> import the_module

> the_module.sys

> would work, but

> from the_module import sys

> would not work?

> That might be odd and confusing.

Good point. I'm not familiar with CPython internals so I'm not sure how this would work on the implementation side, but I definitely think it would be important to not have an inconsistency here.

So I think there is no technical reason why we couldn't make it so that a module that uses `export` returns a proxy that only lets you use attributes that are explicitly exported. If users disagree with this, they can negotiate directly with the module authors. It's not a principle of Python that users *must* be given access to implementation details, after all -- it was just more convenient to do it this way when the language was young. (For example, it's not possible to root around like this in C extensions -- these only export what the author intends to export.)

If this is implemented, then please ensure that some mechanism is included (either keeping __dict__, or functions in inspect, or some other mechanism) to both get and set all attributes of a module irrespective of any export/__all__ controls.

I understand that perspective that says library authors should be able to control their API, for various reasons, but this has to be balanced against the fact that library authors are not perfect, and can either include bugs, or fail to consider all reasonable use-cases when designing their code. Historically, this has been done, by convention, through use of the "_" private specifier for module-level objects.

The value of being able to (in specific cases) reach into third-party code, and customize it to work for your specific situation should not be disregarded.

I think every large codebase that I've worked with has had to monkey-patch a method deep within at least one third-party library at runtime, in production (this is also commonly used for testing purposes), at some point to work-around either a limitation of the library, incompatibility with other library, or to patch a bug. This also applies to module-globals that are themselves imported modules, being able to rebind a name to reference a different module (within careful constraints) in a library module has saved me several times.

If external visibility of code units within libraries starts to be restricted so that this sort of patching isn't possible, then the cost of fixing some problems may start to become significantly greater.

I strongly believe that the access to members of modules should be kept as similar to the access of members of class instances as possible. Not only will this keep the model simpler, the same arguments and/or concerns apply when talking about classes as when talking about modules. Maybe this means that non-exported members of modules get name-mangled as protected class attributes do, or maybe the '__dict__' member is always exported.

I think the idea of resolving the issues mentioned above by talking with third-party module authors directly, seems like an unlikely solution. In my experience, this sort of 'keyhole surgery' patching is often against large, undersupported public libraries (think sqlalchemy, pandas, etc.. where two factors make it unlikely that these projects will be able to help directly:

1. If your project have unique or uncommon requirements that require this sort of patch, often library authors do not want to incur the maintenance burden of changing their carefully-designed code to support the use-cases, especially if they do not consider the requirements to be valid (even if they are in a different context to that being considered by the authors)
2. Release cycles of these projects can be long and complex. Maybe you're using a previous version of this library, and the current release is not compatible with your stack, so even if the library does release a fix, then you'd not be able to use it without a major refactor/retest cycle internally.

I really like the "we're all adults here" approach to private/protected access that python currently takes, please don't weaken this without a lot of serious consideration :)

Thanks

Steve