From sanner@scripps.edu  Wed Mar 14 22:36:56 2001
From: sanner@scripps.edu (Michel Sanner)
Date: Wed, 14 Mar 2001 14:36:56 -0800
Subject: [Import-sig] imputil
Message-ID: <1010314143656.ZM2440@noah.scripps.edu>

Hi,

I am trying to come up with a way to handle versions of packages using imputil.
I got version 1.12 from the CVS server and I try the following:

package structure:

A/
        __init__.py
                from imputil import ImportManager
                ImportManager().install()
        A_1_0/
                __init__.py
                B.py
        A_1_1/
                __init__.py
                B.py
        A_1_2/
                __init__.py
                B.py

Here is what I'd like to achieve:
>>> import A
>>> # the __init__.py file of the A package identifies A_1_0, A_1_1 and A_1_2
>>> # as sub packages and installs a special importer which will import from
>>> # A_VERSION every time that something from A should be imported
>>>
>>> A.importer.set_version("1_0")
>>> from A import B # should import A.A_1_0.B
>>>

The first problem I have is the following:

if I type:

Python 1.5.2 (#0, Apr 14 2000, 10:15:38) [C] on irix646
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import A
>>> from A import A_1_0
Traceback (innermost last):
  File "<stdin>", line 1, in ?
ImportError: cannot import name A_1_0
>>>

if I type "from A import A_1_0" BEFORE importing the package A it works fine

Python 1.5.2 (#0, Apr 14 2000, 10:15:38) [C] on irix646
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> from A import A_1_0
>>> A_1_0
<module 'A.A_1_0' from 'A/A_1_0/__init__.pyc'>
>>>

more generally, I do not understand how imputil should be used.
It seems that I should sub-class Importer and overwrite the get_code method.
How would I then make this importer the one used for my package ?

Thanks for any help with that

-Michel

-- 

-----------------------------------------------------------------------

>>>>>>>>>> AREA CODE CHANGE <<<<<<<<< we are now 858 !!!!!!!

Michel F. Sanner Ph.D.                   The Scripps Research Institute
Assistant Professor			Department of Molecular Biology
					  10550 North Torrey Pines Road
Tel. (858) 784-2341				     La Jolla, CA 92037
Fax. (858) 784-2860
sanner@scripps.edu                        http://www.scripps.edu/sanner
-----------------------------------------------------------------------



From gmcm@hypernet.com  Thu Mar 15 04:16:19 2001
From: gmcm@hypernet.com (Gordon McMillan)
Date: Wed, 14 Mar 2001 23:16:19 -0500
Subject: [Import-sig] imputil
In-Reply-To: <1010314143656.ZM2440@noah.scripps.edu>
Message-ID: <3AAFFBC3.31938.8239B593@localhost>

Michel Sanner wrote:
> 
> I am trying to come up with a way to handle versions of packages
> using imputil. I got version 1.12 from the CVS server and I try
> the following:
> 
> package structure:
> 
> A/
>         __init__.py
>                 from imputil import ImportManager
>                 ImportManager().install()
>         A_1_0/
>                 __init__.py
>                 B.py
>         A_1_1/
>                 __init__.py
>                 B.py
>         A_1_2/
>                 __init__.py
>                 B.py
> 
> Here is what I'd like to achieve:
> >>> import A
> >>> # the __init__.py file of the A package identifies A_1_0,
> >>> # A_1_1 and A_1_2 as sub packages and installs a special
> >>> # importer which will import from A_VERSION every time that
> >>> # something from A should be imported

Maybe an important point: what exactly do you mean by 
"installs a special importer"?

> >>> A.importer.set_version("1_0")
> >>> from A import B # should import A.A_1_0.B
> >>>
> 
> The first problem I have is the following:
> 
> if I type:
> 
> Python 1.5.2 (#0, Apr 14 2000, 10:15:38) [C] on irix646
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>>
> import A >>> from A import A_1_0 Traceback (innermost last):
>   File "<stdin>", line 1, in ?
> ImportError: cannot import name A_1_0
> >>>

ImportManager doesn't do any importing. Something along the 
lines of 
 sys.path.append(A.importer) 
might do the trick.
 
> if I type "from A import A_1_0" BEFORE importing the package A it
> works fine
> 
> Python 1.5.2 (#0, Apr 14 2000, 10:15:38) [C] on irix646
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>>
> from A import A_1_0 >>> A_1_0 <module 'A.A_1_0' from
> 'A/A_1_0/__init__.pyc'> >>>
> 
> more generally, I do not understand how imputil should be used.
> It seems that I should sub-class Importer and overwrite the
> get_code method. How would I then make this importer the one used
> for my package ?

Actually, in this case you don't really *need* imputil. You have 
a normal package structure, and just want to indulge in a little 
indirection. Most people would probably (in A/__init__.py) do 
an import * on the proper version. If it's small enough (or it's all 
going to end up loaded anyway) this is OK.

To do things on demand, you've got a bunch of choices, many 
of them dirty, nasty tricks (such as assigning to 
sys.modules['A'] - 'nuff said about that <wink wink>). You 
could certainly use imputil (but you'll need to put an 
imputil.BuiltinImporter() on sys.path, too). But since you're 
only talking about subpackages, you can do more private 
things. Among the things that happen when someone imports 
A.B is that Python looks in  A for the name B, which is to say 
that it looks in A/__init__.py's module dictionary. So you don't 
really need to provide a whole importer, you just need to 
provide the name B (bound sooner or later to the module 
<version>/B.py). Another choice is to play with the packages 
__path__ attribute in __init__.py (but I'm not fond of that, 
either).

[I am *not* dissing imputil here at all; but imputil affects *all* 
imports once it's installed so you need to be careful of side 
effects.]

- Gordon


From pf@artcom-gmbh.de  Thu Mar 15 07:34:28 2001
From: pf@artcom-gmbh.de (Peter Funk)
Date: Thu, 15 Mar 2001 08:34:28 +0100 (MET)
Subject: versioning schemes in package imports (was Re: [Import-sig] imputil)
In-Reply-To: <3AAFFBC3.31938.8239B593@localhost> from Gordon McMillan at "Mar 14, 2001 11:16:19 pm"
Message-ID: <m14dSHE-000CnFC@artcom0.artcom-gmbh.de>

Hi,

sorry for jumping in on this thread that late.

> Michel Sanner wrote:
> > I am trying to come up with a way to handle versions of packages
[...]
> > package structure:
> > A/
> >         A_1_0/
> >         A_1_1/
> >         A_1_2/
[...]
> > >>> A.importer.set_version("1_0")
> > >>> from A import B # should import A.A_1_0.B
[...]

Gordon McMillan answered:
> Actually, in this case you don't really *need* imputil.
[...]

What I want to add is: The package Pmw (Python Mega Widgets,
see <http://Pmw.sourceforge.net/>.) written by Greg McFarlane
<mailto:gregm@iname.com> contains a versioning mechanism comparable
to what Michel Saner seems to want.

BTW: Pmw is a Python extension very useful for everybody interested
in programming X-platform portable GUIs.

After 'import Pmw' the Pmw/__init__.py file replaces the Pmw module with 
an instance of a very interesting class 'PmwLoader', which fullfills 
two purposes:

1. Select a specific version of Pmw from a subdirectory called
   for example Pmw_0_8_3 or Pmw_0_8_4 or so.
2. Implement a "lazy importer" which delays actual importing of
   submodules contained in the package until a feature contained
   in it is actually used by the application which increases startup 
   performance.

I personally never understood, why people bother with versioning
schemes and focussed on the second idea, which I found very smart.
It was easy for me to rip the versioning stuff out of the PmwLoader
class and to build a my own simplified lazy importer class for use
in my own packages.

May be Michel could use the same approach to implement his own
loader class doing this kind of version selection, he outlined 
above?  The basic idea to do all such things is outlined in the 
last three lines of Pmw/__init__.py which I quote here:

  # Create the dynamic loader and install it into sys.modules.
  sys.modules['_Pmw'] = sys.modules['Pmw']
  sys.modules['Pmw'] = _mod.PmwLoader(_dir, _instdirs, _listdir)

But this approach (replacing the module in sys.modules with a class
instance after import) has some disadvantages: For example: The
new documentation tool pydoc.py written by Ping (now in the standard
library) breaks down when encountering such a "module", because it
has no __file__ attribute.

Regards, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)



From mal@lemburg.com  Thu Mar 15 17:35:01 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Thu, 15 Mar 2001 18:35:01 +0100
Subject: versioning schemes in package imports (was Re: [Import-sig] imputil)
References: <m14dSHE-000CnFC@artcom0.artcom-gmbh.de>
Message-ID: <3AB0FD45.10F56E2@lemburg.com>

Peter Funk wrote:
> 
> Hi,
> 
> sorry for jumping in on this thread that late.
> 
> > Michel Sanner wrote:
> > > I am trying to come up with a way to handle versions of packages
> [...]
> > > package structure:
> > > A/
> > >         A_1_0/
> > >         A_1_1/
> > >         A_1_2/
> [...]
> > > >>> A.importer.set_version("1_0")
> > > >>> from A import B # should import A.A_1_0.B
> [...]
> 
> Gordon McMillan answered:
> > Actually, in this case you don't really *need* imputil.
> [...]
> 
> What I want to add is: The package Pmw (Python Mega Widgets,
> see <http://Pmw.sourceforge.net/>.) written by Greg McFarlane
> <mailto:gregm@iname.com> contains a versioning mechanism comparable
> to what Michel Saner seems to want.
> 
> BTW: Pmw is a Python extension very useful for everybody interested
> in programming X-platform portable GUIs.
> 
> After 'import Pmw' the Pmw/__init__.py file replaces the Pmw module with
> an instance of a very interesting class 'PmwLoader', which fullfills
> two purposes:
> 
> 1. Select a specific version of Pmw from a subdirectory called
>    for example Pmw_0_8_3 or Pmw_0_8_4 or so.
> 2. Implement a "lazy importer" which delays actual importing of
>    submodules contained in the package until a feature contained
>    in it is actually used by the application which increases startup
>    performance.
> 
> I personally never understood, why people bother with versioning
> schemes and focussed on the second idea, which I found very smart.
> It was easy for me to rip the versioning stuff out of the PmwLoader
> class and to build a my own simplified lazy importer class for use
> in my own packages.

You may want to have a look at mx.Misc.LazyModule in the
egenix-mx-base package. It is used by most mx subpackages
to enable submodule/-package load without additional import.

Note that is doesn't try to manipulate sys.modules, but instead
works in the namespace defining the lazy module.

> May be Michel could use the same approach to implement his own
> loader class doing this kind of version selection, he outlined
> above?  The basic idea to do all such things is outlined in the
> last three lines of Pmw/__init__.py which I quote here:
> 
>   # Create the dynamic loader and install it into sys.modules.
>   sys.modules['_Pmw'] = sys.modules['Pmw']
>   sys.modules['Pmw'] = _mod.PmwLoader(_dir, _instdirs, _listdir)
> 
> But this approach (replacing the module in sys.modules with a class
> instance after import) has some disadvantages: For example: The
> new documentation tool pydoc.py written by Ping (now in the standard
> library) breaks down when encountering such a "module", because it
> has no __file__ attribute.
> 
> Regards, Peter
> --
> Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
> office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)
> 
> _______________________________________________
> Import-sig mailing list
> Import-sig@python.org
> http://mail.python.org/mailman/listinfo/import-sig

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Pages:                           http://www.lemburg.com/python/


From sanner@scripps.edu  Thu Mar 15 18:21:40 2001
From: sanner@scripps.edu (Michel Sanner)
Date: Thu, 15 Mar 2001 10:21:40 -0800
Subject: versioning schemes in package imports (was Re: [Import-sig] imputil)
Message-ID: <1010315102140.ZM1987@noah.scripps.edu>

Sorry MAL,

I really meant to send this to import-sig@python.org

Thanks you all for these useful information

On Mar 15,  6:35pm, M.-A. Lemburg wrote:
> Subject: Re: versioning schemes in package imports (was Re: [Import-sig] i
> Peter Funk wrote:
> >
> > Hi,
> >
> > sorry for jumping in on this thread that late.
> >
> > > Michel Sanner wrote:
> > > > I am trying to come up with a way to handle versions of packages
> > [...]
> > > > package structure:
> > > > A/
> > > >         A_1_0/
> > > >         A_1_1/
> > > >         A_1_2/
> > [...]
> > > > >>> A.importer.set_version("1_0")
> > > > >>> from A import B # should import A.A_1_0.B
> > [...]
> >
> > Gordon McMillan answered:
> > > Actually, in this case you don't really *need* imputil.
> > [...]
> >
> > What I want to add is: The package Pmw (Python Mega Widgets,
> > see <http://Pmw.sourceforge.net/>.) written by Greg McFarlane
> > <mailto:gregm@iname.com> contains a versioning mechanism comparable
> > to what Michel Saner seems to want.
> >
> > BTW: Pmw is a Python extension very useful for everybody interested
> > in programming X-platform portable GUIs.
> >

We are using Pmw and I looked at its importer. What I did not like is that each
package would have to implement its own importer. I have an application Pmv
that uses about 7 other packages. I could not see how to re-implement the
PmwLoader to make it a generic loader that all these packages could use.

The other problem I noticed was that I cannot say:
>>> from Pmw import EntryField

I have wuite a bit of code that is alreday written using such imports and using
an approach preventing the "from package import something" syntax would mean
major modifications :(

I will look at your "riped-out version" may be you did exactly what I wanted to
do but did not know how to ! (could you point me towards one of these packages
?)

>
> You may want to have a look at mx.Misc.LazyModule in the
> egenix-mx-base package. It is used by most mx subpackages
> to enable submodule/-package load without additional import.
>
I will look at the mx.Misc.LazyModule


On a more general note, I use Python because it allows me to develop re-usable
components and I do re-sue other people's components. This is a major strength
of Python in my opinion. If every package has its own way to deal with versions
it makes it complicated and less elegant to check for version and set versions.
It would be great if versioning was an integral part of packages and would be
handled the same way by everyone. maybe like in Tcl where you can

require pacakage foo 1.5

Thanks again for the hints. I will drop imputil and investigated the different
alternative you have suggested.

-Michel

--

-----------------------------------------------------------------------

>>>>>>>>>> AREA CODE CHANGE <<<<<<<<< we are now 858 !!!!!!!

Michel F. Sanner Ph.D.                   The Scripps Research Institute
Assistant Professor			Department of Molecular Biology
					  10550 North Torrey Pines Road
Tel. (858) 784-2341				     La Jolla, CA 92037
Fax. (858) 784-2860
sanner@scripps.edu                        http://www.scripps.edu/sanner
-----------------------------------------------------------------------



---End of forwarded mail from <sanner@noah.scripps.edu> ("Michel Sanner")

-- 

-----------------------------------------------------------------------

>>>>>>>>>> AREA CODE CHANGE <<<<<<<<< we are now 858 !!!!!!!

Michel F. Sanner Ph.D.                   The Scripps Research Institute
Assistant Professor			Department of Molecular Biology
					  10550 North Torrey Pines Road
Tel. (858) 784-2341				     La Jolla, CA 92037
Fax. (858) 784-2860
sanner@scripps.edu                        http://www.scripps.edu/sanner
-----------------------------------------------------------------------



From sanner@scripps.edu  Sat Mar 17 01:43:23 2001
From: sanner@scripps.edu (Michel Sanner)
Date: Fri, 16 Mar 2001 17:43:23 -0800
Subject: [Import-sig] import question
Message-ID: <1010316174323.ZM10134@noah.scripps.edu>

Hi, I didn't get any response on help-python.org so I figured I try these lists


if I have the follwoing packages hierarchy

A/
	__init__.py
        B/
		__init__.py
		C.py


I can use:

>>> from A.B import C

but if I use:

>>> import A
>>> print A
<module 'A' from 'A/__init__.pyc'>
>>> from A import B
print B
<module 'A.B' from 'A/B/__init__.py'>
>>> from B import C
Traceback (innermost last):
  File "<stdin>", line 1, in ?
ImportError: No module named B

in order to get this to work I have to

>>> import sys
>>> sys.modules['B'] = B

Is that expected ?
In the documentation I read:

"from" module "import" identifier

so I expected "from B import C" to be legal since B is a module

I tried this with Python 1.5.2 and 2.0 on an sgi under IRIX6.5

Thanks for any help

-Michel

-- 

-----------------------------------------------------------------------

>>>>>>>>>> AREA CODE CHANGE <<<<<<<<< we are now 858 !!!!!!!

Michel F. Sanner Ph.D.                   The Scripps Research Institute
Assistant Professor			Department of Molecular Biology
					  10550 North Torrey Pines Road
Tel. (858) 784-2341				     La Jolla, CA 92037
Fax. (858) 784-2860
sanner@scripps.edu                        http://www.scripps.edu/sanner
-----------------------------------------------------------------------



From guido@digicool.com  Sat Mar 17 02:13:14 2001
From: guido@digicool.com (Guido van Rossum)
Date: Fri, 16 Mar 2001 21:13:14 -0500
Subject: [Import-sig] import question
In-Reply-To: Your message of "Fri, 16 Mar 2001 17:43:23 PST."
 <1010316174323.ZM10134@noah.scripps.edu>
References: <1010316174323.ZM10134@noah.scripps.edu>
Message-ID: <200103170213.VAA13856@cj20424-a.reston1.va.home.com>

> if I have the follwoing packages hierarchy
> 
> A/
> 	__init__.py
>         B/
> 		__init__.py
> 		C.py
> 
> 
> I can use:
> 
> >>> from A.B import C
> 
> but if I use:
> 
> >>> import A
> >>> print A
> <module 'A' from 'A/__init__.pyc'>
> >>> from A import B
> print B
> <module 'A.B' from 'A/B/__init__.py'>
> >>> from B import C
> Traceback (innermost last):
>   File "<stdin>", line 1, in ?
> ImportError: No module named B
> 
> in order to get this to work I have to
> 
> >>> import sys
> >>> sys.modules['B'] = B
> 
> Is that expected ?
> In the documentation I read:
> 
> "from" module "import" identifier
> 
> so I expected "from B import C" to be legal since B is a module
> 
> I tried this with Python 1.5.2 and 2.0 on an sgi under IRIX6.5
> 
> Thanks for any help
> 
> -Michel

In "from X import Y", X is not a reference to a name in your
namespace, it is a module name.  The right thing is indeed to write
"from A.B import C".  There's no way to shorten this; what you did
(assigning sys.modules['B'] = B) is asking for trouble.

Sorry!

--Guido van Rossum (home page: http://www.python.org/~guido/)


From sanner@scripps.edu  Fri Mar 23 18:56:11 2001
From: sanner@scripps.edu (Michel Sanner)
Date: Fri, 23 Mar 2001 10:56:11 -0800
Subject: [Import-sig] import with versions
Message-ID: <1010323105612.ZM39214@noah.scripps.edu>

Hi,

and thank you to all of you who answered my previous email.

here is what I came up with:

def setVersion(version):
    import os, sys

    # get the generic package name (no version number, say A)
    modDir, modName = os.path.split(__path__[0])

    # build the name of the sub package with version (A_1_0)
    modVersName = modName+'_'+version

    # build the fully qualified package name (A.A_1_0)
    fqModVersName = modName+'.'+modVersName

    # import the sub package DejaVu.DejaVu_1_0 and get a handle to in modVers
    mod = __import__(fqModVersName)
    modVers = getattr(mod, modVersName)

    # temporarly replace the generic module by the module with a version number
    # to enable from A.B import C as all modules of A.A_1_0 are pulled in
    import sys
    sys.modules[modName]=modVers


    # import everything from the module with version. This relies on the
    # __init__.py of the module with version to define __all__.
    modVers = __import__( fqModVersName, locals(), globals(), ['*'] )

    # restor generic module
    sys.modules[modName]=mod

    # loop over the imported objects and added them to the generic
    # module's name space to make "A.A_1_0.B()" valid
    # If the value is a module we also added it to sys.modules to make
    # "from A.A_1_0 import B" valid
    for k,v in modVers.__dict__.items():
        if k[:2]=='__': continue
        setattr(mod,k, v)
        if type(v)==types.ModuleType:
            sys.modules[modName+'.'+k] = v

I place this function in the __init__.py file of my package A.I can then say

>>> import A
>>> A.setVersion('1_0')
>>> from A import B
>>> B
>>> <module 'A.A_1_0.B' from 'A.A_1_0.B.py'>

so this works ! :)

The only problem I could not resolve is the following:

if B.py contains:

>>> from A.C import D

and E.py contains

>>> from C import D

these two classes end up being different. the first one will be
<class A.C.D at 102495e8>
while the second one will be
<class A.A_1_0.C.D at 1025ff40>

In other words this can only work if all imports within the package are done in
a "relative" way ie "from C import D" rather than using a fully qualified name
ie "from A.C import D".

as a consequence many "isinstance(a, b)" test in my code break depending on
whether the class b has been imported in a relative or an absolute way.

I do not understand the details of importing enough to know if this can be
worked around. Suggestions and comments are welcome :)

It would be great to be able to have multiple versions of a package online at
the same time and it should not limit the language's syntax (as some versioning
schemes described earlier do, for instnce not allowing the from package import
.. statment).

-Michel

-- 

-----------------------------------------------------------------------

>>>>>>>>>> AREA CODE CHANGE <<<<<<<<< we are now 858 !!!!!!!

Michel F. Sanner Ph.D.                   The Scripps Research Institute
Assistant Professor			Department of Molecular Biology
					  10550 North Torrey Pines Road
Tel. (858) 784-2341				     La Jolla, CA 92037
Fax. (858) 784-2860
sanner@scripps.edu                        http://www.scripps.edu/sanner
-----------------------------------------------------------------------