[Neuroimaging] Technical details managing Python versions and packages.

[Matthew Brett]

Hi,

On Fri, Jul 31, 2015 at 6:57 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Hi Omar,
>
> On Fri, Jul 31, 2015 at 6:29 PM, Jesus-Omar Ocegueda-Gonzalez
> <jomaroceguedag at gmail.com> wrote:
>> Hello Python experts!,
>> I just wanted to ask if anyone of you could point me out to a good reference
>> to learn a good way to manage different python versions and their
>> corresponding packages. This is a bit embarrassing, but I guess the
>> following story may seem familiar to some people (probably those days when
>> you were python newbies): I had python 2.7 with a lot of packages already
>> installed, some of them installed with pip, some with easy_install, some
>> with apt-get, some built manually from source code... who knows? I just
>> sequentially tried each installation way, following instructions I found in
>> random internet pages whenever something went wrong, until one of the
>> installation instructions suddenly worked, and just moved on. Then, at some
>> point, I tried to install Python 3 to reproduce a bug reported to only
>> happen there, just to discover that now nothing works, I have no numpy, no
>> nibabel, none of the basic packages, so I tried to "re-install" them
>> (following instructions from random internet pages when something goes
>> wrong... again), see the pattern?. So the root cause is obviously that I
>> have no idea of what's going on behind scenes when I use these "mysterious"
>> installers, and how they affect my environment, which of them are compatible
>> with each other and which are not, etc.
>>
>> So, to break the pattern, I think this is time to really learn exactly
>> what's going on when we "install" packages with different tools, and how to
>> correctly manage different versions. Could anyone point me out to a good
>> reference to learn these details (e.g. Is there a good way to actually
>> remove everything so we can start a totally fresh installation)?
>>
>> Thank you very much in advance!.
>> With warm regards,
>> -A frustrated -but motivated- Python user.
>
> I'm afraid I don't know of a very good reference.
>
> I think your two options are:
>
> * start from scratch using virtualenv and pip;
> * use conda
>
> I prefer virtualenv and pip, because want to support standard Python
> packaging, and am worried about the effect of handing over Python
> packaging to one company.  Although conda is open-source, almost all
> the work and all the hosting is done by employees of Continuum IO.
>
> Having said that, I have the impression that conda is a reliable
> solution to the problem of having multiple different Python
> environments you want to work on.
>
> If you want to go the virtualenv / pip route, I recommend you start
> from scratch.  In particular, easy_install had a very nasty trick of
> putting installed packages unconditionally at the top of the Python
> search path, using `easy-install.pth` files.
>
> If you're interested to do that, let me know, I'll write up a summary
> of what to do.  I guess you are on Debian / Ubuntu?

Despite deafening silence, I did write up the summary:

https://matthew-brett.github.io/pydagogue/installing_on_debian.html

Cheers,

Matthew