[Neuroimaging] Technical details managing Python versions and packages.

[Matthew Brett]

Hi Omar,

On Fri, Jul 31, 2015 at 6:29 PM, Jesus-Omar Ocegueda-Gonzalez
<jomaroceguedag at gmail.com> wrote:
> Hello Python experts!,
> I just wanted to ask if anyone of you could point me out to a good reference
> to learn a good way to manage different python versions and their
> corresponding packages. This is a bit embarrassing, but I guess the
> following story may seem familiar to some people (probably those days when
> you were python newbies): I had python 2.7 with a lot of packages already
> installed, some of them installed with pip, some with easy_install, some
> with apt-get, some built manually from source code... who knows? I just
> sequentially tried each installation way, following instructions I found in
> random internet pages whenever something went wrong, until one of the
> installation instructions suddenly worked, and just moved on. Then, at some
> point, I tried to install Python 3 to reproduce a bug reported to only
> happen there, just to discover that now nothing works, I have no numpy, no
> nibabel, none of the basic packages, so I tried to "re-install" them
> (following instructions from random internet pages when something goes
> wrong... again), see the pattern?. So the root cause is obviously that I
> have no idea of what's going on behind scenes when I use these "mysterious"
> installers, and how they affect my environment, which of them are compatible
> with each other and which are not, etc.
>
> So, to break the pattern, I think this is time to really learn exactly
> what's going on when we "install" packages with different tools, and how to
> correctly manage different versions. Could anyone point me out to a good
> reference to learn these details (e.g. Is there a good way to actually
> remove everything so we can start a totally fresh installation)?
>
> Thank you very much in advance!.
> With warm regards,
> -A frustrated -but motivated- Python user.

I'm afraid I don't know of a very good reference.

I think your two options are:

* start from scratch using virtualenv and pip;
* use conda

I prefer virtualenv and pip, because want to support standard Python
packaging, and am worried about the effect of handing over Python
packaging to one company.  Although conda is open-source, almost all
the work and all the hosting is done by employees of Continuum IO.

Having said that, I have the impression that conda is a reliable
solution to the problem of having multiple different Python
environments you want to work on.

If you want to go the virtualenv / pip route, I recommend you start
from scratch.  In particular, easy_install had a very nasty trick of
putting installed packages unconditionally at the top of the Python
search path, using `easy-install.pth` files.

If you're interested to do that, let me know, I'll write up a summary
of what to do.  I guess you are on Debian / Ubuntu?

Cheers,

Matthew