Importing public symbols and simultainiously privatizing them, is too noisy
Has anyone else found this to be too syntactically noisy? from module import Foo as _Foo, bar as _bar That is horrifically noisy IMO. The problem is, how do we remove the noise without sacrificing intuitiveness? My first idea was to do this: from module import_private Foo, bar And while it's self explanatory, it's also too long. So i thought... from module _import Foo, bar I'm leaning more towards the latter, but i'm not loving it either. Any ideas?
Rick Johnson <rantingrickjohnson@gmail.com> writes:
Has anyone else found this to be too syntactically noisy?
from module import Foo as _Foo, bar as _bar
That is horrifically noisy IMO.
Agreed. What is wrong with:: import lorem as _lorem do_something_with(_lorem.Foo) -- \ “Nothing is so common as to imitate one's enemies, and to use | `\ their weapons.” —Voltaire, _Dictionnaire Philosophique_ | _o__) | Ben Finney
On Wednesday, March 16, 2016 at 6:39:01 PM UTC-5, Ben Finney wrote:
Agreed. What is wrong with:
import lorem as _lorem do_something_with(_lorem.Foo)
Yes, in the case when multiple symbols are imported+privatized, that's an improvement, and hiding multiple symbols behind a single symbol *CAN* help to avoid name clashes as well (less names, less chances of clashing) but, in essence, all you've done is to move "syntactic noise" from a single location, to a minimum of one other location, and, potentially, many diverse locations. So yes, useful technique, but not one that fulfills my current "selfish desires" -- I need something sweet baby! O:-)
On Mar 16, 2016, at 18:13, Rick Johnson <rantingrickjohnson@gmail.com> wrote:
On Wednesday, March 16, 2016 at 6:39:01 PM UTC-5, Ben Finney wrote:
Agreed. What is wrong with:
import lorem as _lorem do_something_with(_lorem.Foo)
Yes, in the case when multiple symbols are imported+privatized, that's an improvement, and hiding multiple symbols behind a single symbol *CAN* help to avoid name clashes as well (less names, less chances of clashing) but, in essence, all you've done is to move "syntactic noise" from a single location, to a minimum of one other location, and, potentially, many diverse locations. So yes, useful technique, but not one that fulfills my current "selfish desires" -- I need something sweet baby! O:-)
This seems like something that shouldn't be done in general, so doesn't need a language fix--but if you for some specific reason need to do it all the time, just write a function for it: def _postimport(mod, names): g = sys._getframe(1).f_globals for name in names.split(): g['_'+name] = mod[name] So: import thingy _postimport(thingy, 'spam eggs cheese') use(_spam, _cheese) Of course you can wrap up the import and _postimport in a single function: _magic_from_import('thingy', 'spam eggs cheese') Or use MacroPy to make the syntax nicer. Or, if even that's not good enough, write a simple token-processing import hook that finds "_import" tokens and converts them to calls to your wrapper function. Of course the farther you go down this path, the less readable your code becomes to someone who doesn't know about your function/macro/hook, but if there's really so much boilerplate that it's getting in the way, the tradeoff might be worth it.
On Thu, Mar 17, 2016 at 12:31 PM, Andrew Barnert via Python-ideas <python-ideas@python.org> wrote:
This seems like something that shouldn't be done in general, so doesn't need a language fix--but if you for some specific reason need to do it all the time, just write a function for it:
def _postimport(mod, names): g = sys._getframe(1).f_globals for name in names.split(): g['_'+name] = mod[name]
So:
import thingy _postimport(thingy, 'spam eggs cheese') use(_spam, _cheese)
Of course you can wrap up the import and _postimport in a single function:
_magic_from_import('thingy', 'spam eggs cheese')
Or use MacroPy to make the syntax nicer.
Or, if even that's not good enough, write a simple token-processing import hook that finds "_import" tokens and converts them to calls to your wrapper function.
Of course the farther you go down this path, the less readable your code becomes to someone who doesn't know about your function/macro/hook, but if there's really so much boilerplate that it's getting in the way, the tradeoff might be worth it.
... and then you decide, hey, let's just define __all__ and not worry about adorning all those names with underscores, because the definition of public vs private can be independent of the symbols themselves. ChrisA
On Wednesday, March 16, 2016 at 8:54:06 PM UTC-5, Chris Angelico wrote:
... and then you decide, hey, let's just define __all__ and not worry about adorning all those names with underscores, because the definition of public vs private can be independent of the symbols themselves.
I avoid __all__ like the plague. Too easy for it to get out of sync with the API when i forget to add a new symbol.
On Wed, Mar 16, 2016 at 06:58:55PM -0700, Rick Johnson wrote:
I avoid __all__ like the plague. Too easy for it to get out of sync with the API when i forget to add a new symbol.
In that case why not simply also avoid 'from .. import *' like the plague? It seems far simpler than renaming every single import, which places a permanent burden on future maintainers, code completing editors, grep, etc. FWIW I'd probably _break _down _and cry _if _faced _with _maintaining _code _like _that. David
On Wednesday, March 16, 2016 at 9:19:52 PM UTC-5, David Wilson wrote:
On Wed, Mar 16, 2016 at 06:58:55PM -0700, Rick Johnson wrote:
I avoid __all__ like the plague. Too easy for it to get out of sync with the API when i forget to add a new symbol.
In that case why not simply also avoid 'from .. import *' like the plague?
I do.
It seems far simpler than renaming every single import, which places a permanent burden on future maintainers, code completing editors, grep, etc.
Oh, i understand your confusion now. You think i'm importing *EVERY* symbol this way. No, i'm only importing the symbols i need. And, just FYI, not all modules exist as a "top level component". Many times, you need to split parts of a "component hierarchy" across multiple modules, with each module in the "chain", building upon the last, to create a complex system, that is greater than the sum of all it's parts. For instance: I might start with a "base module" that exposes a few public symbols. Granted, the module may have many private variables, but it only exposes a handful to the public. If i then "inherit" from that module, and "extend it", the symbols that i import from the first module may have made sense to be public in the first module, but now, in the new module, they are merely sub-components of a larger interface, and should therefore be private. (i use the terms "extend", and "inherit" very loosely here) If i simply wanted *ALL* the pubic symbols, and the module i was importing *FROM* followed the Python style guide by using a single underscore for private symbols, then "from module import *" is the way to go -- because i would *ONLY* receive the public symbols.
FWIW I'd probably _break _down _and cry _if _faced _with _maintaining _code _like _that.
This reminds me of a funny story. I once told my doctor "When i do this, it hurts". And you what he told me? He said: "Well, don't do that"
On Wednesday, March 16, 2016 at 8:32:12 PM UTC-5, Andrew Barnert via Python-ideas wrote:
This seems like something that shouldn't be done in general, so doesn't need a language fix--but if you for some specific reason need to do it all the time, just write a function for it:
def _postimport(mod, names): g = sys._getframe(1).f_globals for name in names.split(): g['_'+name] = mod[name]
So:
import thingy _postimport(thingy, 'spam eggs cheese') use(_spam, _cheese)
Of course you can wrap up the import and _postimport in a single function:
_magic_from_import('thingy', 'spam eggs cheese')
Or use MacroPy to make the syntax nicer.
Or, if even that's not good enough, write a simple token-processing import hook that finds "_import" tokens and converts them to calls to your wrapper function.
Of course the farther you go down this path, the less readable your code becomes to someone who doesn't know about your function/macro/hook, but if there's really so much boilerplate that it's getting in the way, the tradeoff might be worth it.
I like your ideas, and thanks for taking the time to write up a few lines of code. Looks like this will have to be a feature that i implement for myself. Heck, I had already planned on re-implementing the entire import mechanism anyway, because I like to spread my module source code across multiple files, and i want to explicitly define *HOW* those source files are combined to create namespaces at run-time, so that i can keep them separate, whist easily sharing state between them.. Python does not allow me to do this without resorting to "import contortions" and monkey patching -- but i know how to do it! *evil grin*
On Mar 16, 2016 21:38, "Rick Johnson" <rantingrickjohnson@gmail.com> wrote:
Heck, I had already planned on re-implementing the entire import mechanism anyway, because I like to spread my module source code across multiple files, and i want to explicitly define *HOW* those source files are combined to create namespaces at run-time, so that i can keep them separate, whist easily sharing state between them.. Python does not allow me to do this without resorting to "import contortions" and monkey patching -- but i know how to do it! *evil grin*
So like Go does packages? You should be able to do that using a custom loader and a path-entry finder to provide that loader. See the importlib docs for more info. Just be careful to only handle your special directories or else normal packages will break. FWIW, there are more idiomatic ways to combine multiple files into a single module. For instance, make a package containing your separate files but make the files private (leading underscore). Then put "from XXX import *" for each of them. If you want to limit what's exported from the package, either define __all__ in the sub-files or use "from XXX import YYY" in the package __init__.py. As far as sharing a namespace between the sub-files, I'd recommend against an implicit mechanism (like Go does). I've found that it makes it harder to discover code. -eric
On Thursday, March 17, 2016 at 12:25:36 PM UTC-5, Eric Snow wrote:
So like Go does packages? You should be able to do that using a custom loader and a path-entry finder to provide that loader. See the importlib docs for more info. Just be careful to only handle your special directories or else normal packages will break.
Yes, i had planned to do it in a manner that will exist side by with the existing "official import mechanism". It's not something i plan to do for "every single module". Only those that have become "too large to easily edit".
FWIW, there are more idiomatic ways to combine multiple files into a single module. For instance, make a package containing your separate files but make the files private (leading underscore). Then put "from XXX import *" for each of them. If you want to limit what's exported from the package, either define __all__ in the sub-files or use "from XXX import YYY" in the package __init__.py.
Sure. That will allow me to "spread-out a large module's source code into N smaller files", and then let Python "combine the objects, defined in those multiple files, under one monolithic namespace at run-time"... but i can't define shared state between them *UNTIL* run-time -- at least, not without resorting to "import spider webs" and monkey patching. I'm not trying to "define a module from N objects contained in N source files", no, i'm trying to "define a *NAMESPACE* from N source files". "N source files" that would transparently share state just as though all the source existed in the exact same file.
As far as sharing a namespace between the sub-files, I'd recommend against an implicit mechanism (like Go does). I've found that it makes it harder to discover code.
Hmm, you're not the only person that has raised this issue. I'm unaware of how Go handles namespaces, but i've used other languages that *DO* offer a programmer the option of "explicitly defining module namespace", and after writing code for many years, I have not once encountered the issues that you and others raise. I'm speculating here, but it may be because i'm very OCD about creating strong interfaces, and i avoid writing code that rebinds symbols "in the shadows". Of course, i'm not suggesting that you, or anyone else, is incompetent, I'm merely pondering, because it does sound like a legitimate possibility... Hey, i can tell you from personal experience: there is nothing worst than implementing a "grand idea", only to find out later, that it was "fools gold". O:-)
Rick Johnson <rantingrickjohnson@gmail.com> writes:
On Wednesday, March 16, 2016 at 6:39:01 PM UTC-5, Ben Finney wrote:
Agreed. What is wrong with:
import lorem as _lorem do_something_with(_lorem.Foo)
Yes, in the case when multiple symbols are imported+privatized, that's an improvement
It's got the strong advantage of already being an established idiom.
but not one that fulfills my current "selfish desires" -- I need something sweet baby! O:-)
You're in the wrong forum, then. This is for discussing improvements to Python. -- \ “We suffer primarily not from our vices or our weaknesses, but | `\ from our illusions.” —Daniel J. Boorstin, historian, 1914–2004 | _o__) | Ben Finney
On Wed, Mar 16, 2016 at 03:50:53PM -0700, Rick Johnson wrote:
from module _import Foo, bar
That's anything but self-explanatory, since it gives no hint that Foo and bar will be imported under some name other than Foo and bar. Besides, why is "import x as _x" so special to require special syntax? Out of the infinite number of names that x could be imported as (import x as y etc) what's so special about _x that it deserves a short-cut? -- Steve
On Wednesday, March 16, 2016 at 7:43:05 PM UTC-5, Steven D'Aprano wrote:
On Wed, Mar 16, 2016 at 03:50:53PM -0700, Rick Johnson wrote:
from module _import Foo, bar
That's anything but self-explanatory, since it gives no hint that Foo and bar will be imported under some name other than Foo and bar.
Yes, I'll agree it would be far too esoteric for a language that promotes readability. If i end up writing my own hack though, i'll probably go this way, simply because it's the most compact solution i think of at the moment.
Besides, why is "import x as _x" so special to require special syntax?
Well, for a single symbol, it's not bad. But consider something like: from module import Foo as _Foo, bar as _bar, BAZ as _BAZ, spam as _spam, eggs as _eggs Now, that may seem like a contrived example, but i've witnessed much longer "run-on import lines" than that. 79 chars will be eaten-up really quickly. But what eats up most of the space, is the redundant " as _...". When importing "piecemeal", it's impossible to avoid writing the actual symbol names , but, if we can avoid the redundant repetition of " as _...", it would be an improvement.
Out of the infinite number of names that x could be imported as (import x as y etc) what's so special about _x that it deserves a short-cut?
Not exactly sure what your saying here, but, i'll fancy a guess... If you're suggesting that my syntax would be limited to merely "adding a single leading underscore to the public symbol (or symbols) as they are imported", then yes, you are correct! The intended purpose is to: "automate the privatization of public symbols during the import process". Of course, if they already have a leading underscore, they will remain unchanged. But i wonder what should be done when they have more than one underscore? Hmm...
From my POV, I don't see anything wrong with "single purpose utilities". In fact, i find them to be more intuitive. The whole: "Do one thing, and do it well" philosophy...
On Wed, Mar 16, 2016 at 06:52:13PM -0700, Rick Johnson wrote:
On Wednesday, March 16, 2016 at 7:43:05 PM UTC-5, Steven D'Aprano wrote:
Out of the infinite number of names that x could be imported as (import x as y etc) what's so special about _x that it deserves a short-cut?
Not exactly sure what your saying here, but, i'll fancy a guess...
"import as" is a general mechanism for renaming variables on import. When can use any new name we like: from module import x as a from module import x as b from module import x as c ... from module import x as z and all get treated the same way: whatever name you want to rename x as, you use the same, standard syntax: "import oldname as newname". But you are suggesting a new keyword, or other built-in functionality, to handle *one special case only*: import oldname as _oldname The Zen of Python tells us about special cases: Special cases aren't special enough to break the rules. Of course, the Zen of Python isn't absolute. If there really is something special enough to justify a special case, then the "Special cases" koan doesn't apply. Hence my question: why is _oldname so special that it deserves special syntax? I'm not questioning that it's special. I'm questioning what makes it sufficiently special that it overrides the design principles espoused by the Zen of Python. -- Steven
On Wed, Mar 16, 2016 at 6:52 PM, Rick Johnson <rantingrickjohnson@gmail.com> wrote:
Besides, why is "import x as _x" so special to require special syntax?
It's not :-) I know I do, for instance, from matplotlib import pylot as plt But have NEVER done the leading underscore thing...
from module import Foo as _Foo, bar as _bar, BAZ as _BAZ, spam as _spam, eggs as _eggs
if you are mirroring an entire namespace, or a god fraction of one then use a module name! import module as _mod then use _mod.Foo, etc..... Now, that may seem like a contrived example, but i've
witnessed much longer "run-on import lines" than that.
I have too, but I think it's bad style -- if you are importing a LOT of names from one module, just import the darn module -- giving it a shorter name if you like. This has become a really standard practice, like: import numpy as np for instance. The intended purpose is to: "automate the privatization of
public symbols during the import process".
I'm really confused about the use case for "privatization of public symbols" at all, but again, if you need a lot of them, use the module name to prefix them. Heck give it a one character name, and then it's hardly more typing than the underscore... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
I avoid __all__ like the plague. Too easy for it to get out of sync with the API when i forget to add a new symbol.
Your API should be one of the most stable parts of your code, no? On Fri, Mar 18, 2016 at 4:29 PM, Chris Barker <chris.barker@noaa.gov> wrote:
On Wed, Mar 16, 2016 at 6:52 PM, Rick Johnson < rantingrickjohnson@gmail.com> wrote:
Besides, why is "import x as _x" so special to require special syntax?
It's not :-) I know I do, for instance,
from matplotlib import pylot as plt
But have NEVER done the leading underscore thing...
from module import Foo as _Foo, bar as _bar, BAZ as _BAZ, spam as _spam, eggs as _eggs
if you are mirroring an entire namespace, or a god fraction of one then use a module name!
import module as _mod
then use _mod.Foo, etc.....
Now, that may seem like a contrived example, but i've
witnessed much longer "run-on import lines" than that.
I have too, but I think it's bad style -- if you are importing a LOT of names from one module, just import the darn module -- giving it a shorter name if you like. This has become a really standard practice, like:
import numpy as np
for instance.
The intended purpose is to: "automate the privatization of
public symbols during the import process".
I'm really confused about the use case for "privatization of public symbols" at all, but again, if you need a lot of them, use the module name to prefix them. Heck give it a one character name, and then it's hardly more typing than the underscore...
-CHB
--
Christopher Barker, Ph.D. Oceanographer
Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@noaa.gov
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
participants (9)
-
Abe Dillon -
Andrew Barnert -
Ben Finney -
Chris Angelico -
Chris Barker -
David Wilson -
Eric Snow -
Rick Johnson -
Steven D'Aprano