[Python-Dev] Those import related syntax errors again...

[Samuele Pedroni]

Hello.

> > > Does the function that contains the exec also contain another function
> > > or lambda?  If it does and the contained function has references to
> > > non-local variables, the compiler will complain.
> >
> > It appears this is the problem.  The fact that only "__init__.py" was
listed
> > threw me - I have a few of them :)
> >
> > *sigh* - this is a real shame.  IMO, we can't continue to break existing
> > code, even if it is good for me!  People are going to get mighty annoyed -
I
> > am.  And if people on python-dev struggle with some of the new errors, the
> > poor normal users are going to feel even more alienated.
>
> Sigh indeed.  We could narrow it down to only raise the error if there
> are nested functions or lambdas that don't reference free variables,
> but unfortunately most of them will reference at least some builtin
> e.g. str()...
>
> How about the old fallback to using straight dict lookups when this
> combination of features is detected?

I'm posting an opinion on this subject because I'm implementing
nested scopes in jython.

It seems that we really should avoid breaking code using import * and exec,
and to obtain this - I agree - the only way is to fall back to some straight
dictionary lookup, when both import or exec and nested scopes are there

But doing this AFAIK related to actual python nested scope impl and what I'm
doing on jython side is quite messy, because we will need to keep around
"chained"
closures as entire dictionaries, because we don't know if an exec or import
will
hide some variable from an outer level, or add a new variable that then
cannot be interpreted as a global one in nested scopes. This is IMO too much
heavyweight.

Another way is to use special rules
(similar to those for class defs), e.g. having

<frag>
y=3
def f():
  exec "y=2"
  def g():
   return y
  return g()

print f()
</frag>

# print 3.

Is that confusing for users? maybe they will more naturally expect 2 as outcome
(given nested scopes).

The last possibility (but I know this one has been somehow discarded) is to
have scoping only
if explicitly declared; I imagine something like

<frag>
y=3
def f():
  let y
  exec "y=2"
  def g():
   return y
  return g()

print f()
</frag>

# print 2.

Issues with this:
- with implicit scoping we naturally obtain that nested func defs can call
themself recursively:
  * we can require a let for this too
  * we can introduce "horrible" things like 'defrec' or 'deflet'
  * we can have def imply a let: this breaks
      def get_str():
        def str(v):
          return "str: "+str(v)
       return str
   but nested scopes as actually implemented already break that.
 - with this approach inner scopes can change the value of outer scope vars:
this was considered a non-feature...
 - what's the gain with this approach? if we consider code like this:
  def f(str): # eg str = "y=z"
   from foo import *
   def g():
     exec str
     return y
  return g
without explicit 'let' decls if we want to compile this and not just say "you
can't do that" the closure of g should be constructed out of the entire runtime
namespace of f.
With explicit 'let's in this case we would produce just the old code and
semantic.
If some 'let' would be added to f, we would know what part of the namespace of
f should be used to construct the closure
of g.
In absence of import* and exec we could use the current fast approach to
implement nested scopes,
if they are there we would know what vars should be stored in cells and passed
down to inner scopes.
[We could have special locals dicts that can contain direct values or cells,
and that would do the right
indirect get and set for the cell-case. These dict could also be possibly
returned by "locals()" and that would
be the way to implement exec "spam", just equivalently as exec "spam" in
globals(),locals(). import * would have just the assignement semantic. ]

Very likely I'm missing something, but
from my "external" viewpoint I would have preferred such solution.
IMO maybe it would be good to think about this, because differently as expected
implicit scoping has consequences
that we would better avoid. Is too late for that (having feature freeze)?

regards, Samuele Pedroni.