On Sun, Aug 12, 2012 at 6:07 PM, Eric Snow <ericsnowcurrently@gmail.com> wrote:

On Aug 12, 2012 12:56 PM, "PJ Eby" <pje@telecommunity.com> wrote:
> I'm not sure if this is directly related or not, but making this
> mechanism support custom compilation for new filename suffixes would
> be nice, especially for various e.g. HTML/XML templating systems that
> compile to Python or bytecode.
>
> Specifically, having a way to add a new source suffix (e.g. ".kid",
> ".zpt", etc.) and a matching compilation function, such that it's
> automatically picked up for compilation by both the filesystem and zip
> importers would be awesome. It'd also allow for DSLs and syntax
> experiments using alternative filename extensions.

+1

I'm hacking around this right now for a project I'm working on. I definitely do this through the import system. Inserting a look-alike path hook and monkeypatching the cached path entry finders is not difficult, but certainly fragile and less than ideal.

Why are you doing that? Can't you just use FileFinder with new loaders and file suffixes? Why do you feel the need to much with any cache?

Consequently I've been looking into simple and not-so-simple solutions to making it easier to add new suffixes to be handled. The source/pyc-to-code-to-module path during imports is so prevalent and critical that it may benefit from its own model.

3.4 will expose more of this. The source-to-code method would get you the transformation that PJE wants while 3.3 already has FileFinder handle the finding of the proper files for you. Toss in a method or two to help in parsing byte-compiled files and writing them and most of the process is then exposed.

This is similar to how the path-based import subsystem has its own special-case model. Or source-based imports just need better special-casing in the path-based import subsystem. Perhaps just adding and using sys.source_suffixes as a mapping of suffixes to loader classes.

This is starting to get off-topic, but no more adding stuff to sys in regards to import. There is already too much global state that is preventing good encapsulation and we don't need to make it any worse. The more we add to sys in regards to import the farther we get from any import engine solution that we might want to evolve towards.