[Pythonmac-SIG] Package for building DMG installers from pip requirements

[Matthew Brett]

Hi,

On 9/19/14, Christopher Barker <pythonchb at gmail.com> wrote:
> On Fri, Sep 19, 2014 at 9:03 AM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
>
>> The primary use case is for getting started with a basic package
>> setup.  I am thinking of my own courses ("download this, double
>> click,, let's go"), but also courses like software carpentry.  Here
>> the students may never have used Terminal.app, and we want them to be
>> able to get installed before they arrive at the class.
>>
>
> well yes and no -- for software carpentry in particular, aren't tey all
> about learning some command line tools, etc? in fact, even more than python
> itself?

My impression is that software carpentry have the same requirements as
I do - they want the install done by the time the students show up.
They are indeed going to show them how to do things on the command
line, but they don't assume they already know how to use it before
they arrive, hence the desire for a point-click installer.

> And for my intro to python classes, I start with "you're going to need to
> do at least a little bit at the command line -- let's get you all going
> there. And frankly, that's easier on OS-X than Windows!
>
> That being said -- yes, nifty installers a nice.
>
> I really like Ned's ideas, but:
>
> What can we write it in? If I were doing it, I'd want use wxPython, becase
> that's how I know to write GUIS -- but that's a pretty big dependency for
> the first thing you need to have.
>
> Could we use PyObjC ? is that delivered with Apple's python installs?
>
> Or would it have to be a simple Cocoa app, so that it was small and compact
> and had no non-standad dependencies?
>
> And whols goign t  odo this? clealry Mathew wants somethign he can use now
> (or soon), so if no one has teh time and inclination to get a beta version
> of an installer our soon, then he might as well kee going with can work
> now.

I believe the project works as it is now.  I will probably start with
an automated build / test of a scipy stack installer on travis, when I
get to better internet, maybe in a couple of weeks.

> The truth is, we talked a an app to handle point and click installation of
> eggs years ago, and no one ever wrote one.
>
>> * Download Python, download get-pip.py,
>
> There is talk of the pip bootstrapping code getting into the standard py2
> installers -- is that going to happen with the next release -- that would
> eliminate one annoying step!
>
>> Open terminal, check you your
>
>> installation of Python gives you Python.org Python,
>
>
> That should more or less be done if you installed the python.org build....

Right - the installer just checks that you have in fact already
installed Python.org; if you haven't the installer gives you a
relevant message.

>> run ``python{3}
>
>> get-pip.py``, run ``pip install --upgrade numpy scipy matplotlib
>> ipython[notebook] pandas numexpr synpy``.
>>
>
> It might be nice to have a "scipy-stack" meta-package that would simply
> depend on all these (or most of them) That would be nice on multiple
> platforms.
>
>> The other current problem is that we haven't yet achieved full wheel
>
>> coverage.  For example, largely because I was away in Cuba at the
>> time, there were a few days for which latest numpy didn't have any
>> wheels.
>
>
> yeah, this is too bad -- it really seems that "the way" 'to distribute
> binaries for python packages is as wheels on PyPI -- building an entire
> other infrastructure really sucks.

I hope that when we've ironed out the builds for the wheels, we'll
develop a habit of releasing the wheels at exactly the same time as we
make the full release, probably by hiding the release while uploading
the wheels.  The numpy delay was because of a bug in my automated
builds that Ralf Gommers bravely fixed while I was away.

For a few other wheels, it seems that some webby package maintainers
have some anxiety about wheel installation, and I couldn't reassure
them with tests on every platform I could think of.  I suppose that
will resolve, given time, and even more comprehensive automated
testing.  Maybe it would also be useful to have some definitive
opinion from the pypa on the likely safety of OSX wheels that I could
point to.

>>  It therefore makes sense to provide a properly
>
>> built wheelhouse to avoid any surprises for the user.
>>
>
> sure.
>
>
>> Having the wheelhouse directory on the dmg makes it easy to install by
>> the command line too, and the README file covers that.
>
>
> Is it that hard to point our nifty still-imaginary installation tool at a
> wheelhouse?

I actually have scipy-stack pip requirement files up on the nipy
server, so it would be trivial to do this - something like:

pip install --no-index --upgrade -r
https://nipy.bic.berkeley.edu/scipy_installers/scipy-stack-1.0-plus.txt

I was looking around Berkeley campus to find some more permanent
hosting for this - it would be fairly easy to make a general
wheelhouse for all our scientific course installers, where the course
requirements would be a pip requirements file.  But - I think we need
just a little bit more infrastructure to make this work smoothly...
Any ideas?

>
>> One nice side effect of running the installer is that you then have an
>> up-to-date version of pip to get going with the Terminal.app
>> lifestyle.
>>
>
> indeed -- maybe if nothing else, a point and click installer for pip....
>
> It's very common for open-source pkg installers not to be signed, so
>> most users are used to 'right-click open' for these,
>
>
> I still find most users choke on this step :-(

Sure - it would be great to have a few funded developer accounts to
fix that.  Ned - any chance of something like that from the PSF?

>> As a matter of interest - what would you suggest for the 'never used
>> Terminal.app, need a lot of packages installed for course' use-case?
>
> I give them clear instructions -- pretty much what you had above  --Though
> I could improve them -- here's my latest:
>
> http://codefellows.github.io/sea-f2-python-sept14/supplements/python_for_mac.html

Nice - thanks.

It would be really good if we could come up with a standard page for
this - something like a mcscipy site ?

> [I haven't pushed the source to gitHub for a while -- but I"ll do that if
> anyone want to contribute!]
>
> That is for a class that is general python, no numpy/scipy., etc. And
> looking now, I probably should have  bit more about opening the terminal in
> a dir, etc. I tend to run through some of that in the first class live, and
> had no problems with the class I'm doing now. The Windows folks -- that's
> another story!
>
> It does mean that that first thing they need to do is figure out at least a
> tiny bit about the command line -- but really -- they are going to need
> that anyway.

Yes - if you don't have much they need to install, then you can easily
do it with pip in the class - otherwise the time and bandwidth can be
limiting.

> I did find that the latest OS-X has an (optional, and off by default, I
> think) right click on a dir and get a menu item to open a terminal there --
> that's a good start.
>
> And I point them to:
>
> http://cli.learncodethehardway.org/book/
>
>
>> Both those ideas sound interesting, but I really need something for a
>> course I'm teaching next month.
>>
>
> yeah, there is that!
>
> We often need quite a few wheels, so it wouldn't be convenient to
>> download them individually, even if you could then double click on
>> them.
>>
>
> but pointing them to a wheelhouse isn't too bad, though I can see the
> appeal of something they can JustGetInstalled -- all of it, before learning
> ANYTHING new.

Thanks a lot for the useful reflections.

If y'all do use the wheels2dmg installer and need any new features,
find it doesn't work, please do let me know...

Onward and upward,

Matthew