On Wednesday, March 16, 2016 at 6:39:01 PM UTC-5, Ben Finney wrote:

Agreed. What is wrong with:

import lorem as _lorem
    do_something_with(_lorem.Foo)

Yes, in the case when multiple symbols are imported+privatized, that's an improvement, and hiding multiple symbols behind a single symbol *CAN* help to avoid name clashes as well (less names, less chances of clashing) but, in essence, all you've done is to move "syntactic noise" from a single location, to a minimum of one other location, and, potentially, many diverse locations. So yes, useful technique, but not one that fulfills my current "selfish desires" -- I need something sweet baby! O:-)

On Thu, Mar 17, 2016 at 12:31 PM, Andrew Barnert via Python-ideas python-ideas@python.org wrote:

This seems like something that shouldn't be done in general, so doesn't need a language fix--but if you for some specific reason need to do it all the time, just write a function for it:

def _postimport(mod, names):
    g = sys._getframe(1).f_globals
    for name in names.split():
        g['_'+name] = mod[name]

So:

import thingy
_postimport(thingy, 'spam eggs cheese')
use(_spam, _cheese)

Of course you can wrap up the import and _postimport in a single function:

_magic_from_import('thingy', 'spam eggs cheese')

Or use MacroPy to make the syntax nicer.

Or, if even that's not good enough, write a simple token-processing import hook that finds "_import" tokens and converts them to calls to your wrapper function.

Of course the farther you go down this path, the less readable your code becomes to someone who doesn't know about your function/macro/hook, but if there's really so much boilerplate that it's getting in the way, the tradeoff might be worth it.

... and then you decide, hey, let's just define __all__ and not worry about adorning all those names with underscores, because the definition of public vs private can be independent of the symbols themselves.

ChrisA

On Wed, Mar 16, 2016 at 06:58:55PM -0700, Rick Johnson wrote:

I avoid  __all__ like the plague. Too easy for it to get out of sync with the API when i forget to add a new symbol.

In that case why not simply also avoid 'from .. import *' like the plague? It seems far simpler than renaming every single import, which places a permanent burden on future maintainers, code completing editors, grep, etc.

FWIW I'd probably _break _down _and cry _if _faced _with _maintaining _code _like _that.

David

On Wednesday, March 16, 2016 at 8:32:12 PM UTC-5, Andrew Barnert via Python-ideas wrote:

This seems like something that shouldn't be done in general, so doesn't need a language fix--but if you for some specific reason need to do it all the time, just write a function for it:

def _postimport(mod, names): 
    g = sys._getframe(1).f_globals 
    for name in names.split(): 
        g['_'+name] = mod[name] 

So:

import thingy 
_postimport(thingy, 'spam eggs cheese') 
use(_spam, _cheese) 

Of course you can wrap up the import and _postimport in a single function:

_magic_from_import('thingy', 'spam eggs cheese')

Or use MacroPy to make the syntax nicer.

Or, if even that's not good enough, write a simple token-processing import hook that finds "_import" tokens and converts them to calls to your wrapper function.

Of course the farther you go down this path, the less readable your code becomes to someone who doesn't know about your function/macro/hook, but if there's really so much boilerplate that it's getting in the way, the tradeoff might be worth it.

I like your ideas, and thanks for taking the time to write up a few lines of code. Looks like this will have to be a feature that i implement for myself. Heck, I had already planned on re-implementing the entire import mechanism anyway, because I like to spread my module source code across multiple files, and i want to explicitly define *HOW* those source files are combined to create namespaces at run-time, so that i can keep them separate, whist easily sharing state between them.. Python does not allow me to do this without resorting to "import contortions" and monkey patching -- but i know how to do it! *evil grin*

On Thursday, March 17, 2016 at 12:25:36 PM UTC-5, Eric Snow wrote:

So like Go does packages? You should be able to do that using a custom loader and a path-entry finder to provide that loader. See the importlib docs for more info. Just be careful to only handle your special directories or else normal packages will break.

Yes, i had planned to do it in a manner that will exist side by with the existing "official import mechanism". It's not something i plan to do for "every single module". Only those that have become "too large to easily edit".

FWIW, there are more idiomatic ways to combine multiple files into a single module. For instance, make a package containing your separate files but make the files private (leading underscore). Then put "from XXX import *" for each of them. If you want to limit what's exported from the package, either define __all__ in the sub-files or use "from XXX import YYY" in the package __init__.py.

Sure. That will allow me to "spread-out a large module's source code into N smaller files", and then let Python "combine the objects, defined in those multiple files, under one monolithic namespace at run-time"... but i can't define shared state between them *UNTIL* run-time -- at least, not without resorting to "import spider webs" and monkey patching.

I'm not trying to "define a module from N objects contained in N source files", no, i'm trying to "define a *NAMESPACE* from N source files". "N source files" that would transparently share state just as though all the source existed in the exact same file.

As far as sharing a namespace between the sub-files, I'd recommend against an implicit mechanism (like Go does). I've found that it makes it harder to discover code.

Hmm, you're not the only person that has raised this issue.

I'm unaware of how Go handles namespaces, but i've used other languages that *DO* offer a programmer the option of "explicitly defining module namespace", and after writing code for many years, I have not once encountered the issues that you and others raise.

I'm speculating here, but it may be because i'm very OCD about creating strong interfaces, and i avoid writing code that rebinds symbols "in the shadows". Of course, i'm not suggesting that you, or anyone else, is incompetent, I'm merely pondering, because it does sound like a legitimate possibility...

Hey, i can tell you from personal experience: there is nothing worst than implementing a "grand idea", only to find out later, that it was "fools gold". O:-)

On Wed, Mar 16, 2016 at 5:38 PM, Ben Finney ben+python@benfinney.id.au wrote:

What is wrong with::

import lorem as _lorem

do_something_with(_lorem.Foo)

+1

-eric

On Wednesday, March 16, 2016 at 7:43:05 PM UTC-5, Steven D'Aprano wrote:

On Wed, Mar 16, 2016 at 03:50:53PM -0700, Rick Johnson wrote:

...

from module _import Foo, bar

That's anything but self-explanatory, since it gives no hint that Foo and bar will be imported under some name other than Foo and bar.

Yes, I'll agree it would be far too esoteric for a language that promotes readability. If i end up writing my own hack though, i'll probably go this way, simply because it's the most compact solution i think of at the moment.

Besides, why is "import x as _x" so special to require special syntax?

Well, for a single symbol, it's not bad. But consider something like:

from module import Foo as _Foo, bar as _bar, BAZ as _BAZ, spam as _spam, eggs as _eggs

Now, that may seem like a contrived example, but i've witnessed much longer "run-on import lines" than that. 79 chars will be eaten-up really quickly. But what eats up most of the space, is the redundant " as _...".

When importing "piecemeal", it's impossible to avoid writing the actual symbol names , but, if we can avoid the redundant repetition of " as _...", it would be an improvement.

Out of the infinite number of names that x could be imported as (import x as y etc) what's so special about _x that it deserves a short-cut?

Not exactly sure what your saying here, but, i'll fancy a guess...

If you're suggesting that my syntax would be limited to merely "adding a single leading underscore to the public symbol (or symbols) as they are imported", then yes, you are correct!

The intended purpose is to: "automate the privatization of public symbols during the import process". Of course, if they already have a leading underscore, they will remain unchanged. But i wonder what should be done when they have more than one underscore? Hmm...

From my POV, I don't see anything wrong with "single purpose

utilities". In fact, i find them to be more intuitive. The whole: "Do one thing, and do it well" philosophy...

On Wed, Mar 16, 2016 at 6:52 PM, Rick Johnson rantingrickjohnson@gmail.com wrote:

...

Besides, why is "import x as _x" so special to require special syntax?

It's not :-) I know I do, for instance,

from matplotlib import pylot as plt

But have NEVER done the leading underscore thing...

from module import Foo as _Foo, bar as _bar, BAZ as _BAZ, spam as _spam, eggs as _eggs

if you are mirroring an entire namespace, or a god fraction of one then use a module name!

import module as _mod

then use _mod.Foo, etc.....

Now, that may seem like a contrived example, but i've

witnessed much longer "run-on import lines" than that.

I have too, but I think it's bad style -- if you are importing a LOT of names from one module, just import the darn module -- giving it a shorter name if you like. This has become a really standard practice, like:

import numpy as np

for instance.

The intended purpose is to: "automate the privatization of

public symbols during the import process".

I'm really confused about the use case for "privatization of public symbols" at all, but again, if you need a lot of them, use the module name to prefix them. Heck give it a one character name, and then it's hardly more typing than the underscore...

-CHB