[Python-Dev] advice needed: best approach to enabling "metamodules"?

[Guido van Rossum]

Nathaniel, did you look at Brett's LazyLoader? It overcomes the subclass
issue by using a module loader that makes all modules instances of a
(trivial) Module subclass. I'm sure this approach can be backported as far
as you need to go.

On Sun, Nov 30, 2014 at 5:02 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Mon, Dec 1, 2014 at 12:59 AM, Nathaniel Smith <njs at pobox.com> wrote:
> > On Sun, Nov 30, 2014 at 10:14 PM, Mark Shannon <mark at hotpy.org> wrote:
> >> Hi,
> >>
> >> This discussion has been going on for a while, but no one has
> questioned the
> >> basic premise. Does this needs any change to the language or
> interpreter?
> >>
> >> I believe it does not. I'm modified your original metamodule.py to not
> use
> >> ctypes and support reloading:
> >> https://gist.github.com/markshannon/1868e7e6115d70ce6e76
> >
> > Interesting approach!
> >
> > As written, your code will blow up on any python < 3.4, because when
> > old_module gets deallocated it'll wipe the module dict clean. And I
> > guess even on >=3.4, this might still happen if old_module somehow
> > manages to get itself into a reference loop before getting
> > deallocated. (Hopefully not, but what a nightmare to debug if it did.)
> > However, both of these issues can be fixed by stashing a reference to
> > old_module somewhere in new_module.
> >
> > The __class__ = ModuleType trick is super-clever but makes me
> > irrationally uncomfortable. I know that this is documented as a valid
> > method of fooling isinstance(), but I didn't know that until your
> > yesterday, and the idea of objects where type(foo) is not
> > foo.__class__ strikes me as somewhat blasphemous. Maybe this is all
> > fine though.
> >
> > The pseudo-module objects generated this way will still won't pass
> > PyModule_Check, so in theory this could produce behavioural
> > differences. I can't name any specific places where this will break
> > things, though. From a quick skim of the CPython source, a few
> > observations: It means the PyModule_* API functions won't work (e.g.
> > PyModule_GetDict); maybe these aren't used enough to matter. It looks
> > like the __reduce__ methods on "method objects"
> > (Objects/methodobject.c) have a special check for ->m_self being a
> > module object, and won't pickle correctly if ->m_self ends up pointing
> > to one of these pseudo-modules. I have no idea how one ends up with a
> > method whose ->m_self points to a module object, though -- maybe it
> > never actually happens. PyImport_Cleanup treats module objects
> > differently from non-module objects during shutdown.
>
> Actually, there is one showstopper here -- in the first version where
> reload() uses isinstance() is actually 3.4. Before that you need a
> real module subtype for reload to work. But this is in principle
> workaroundable by using subclassing + ctypes on old versions of python
> and the __class__ = hack on new versions.
>
> > I guess it also has the mild limitation that it doesn't work with
> > extension modules, but eh. Mostly I'd be nervous about the two points
> > above.
> >
> > -n
> >
> > --
> > Nathaniel J. Smith
> > Postdoctoral researcher - Informatics - University of Edinburgh
> > http://vorpus.org
>
>
>
> --
> Nathaniel J. Smith
> Postdoctoral researcher - Informatics - University of Edinburgh
> http://vorpus.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141130/6837ad04/attachment.html>