[Numpy-discussion] help from OS X 10.5 users wanted

[Ralf Gommers]

On Sat, Oct 16, 2010 at 9:32 PM, Friedrich Romstedt
<friedrichromstedt at gmail.com> wrote:
> 2010/10/16 Ralf Gommers <ralf.gommers at googlemail.com>:
>> On Sat, Oct 16, 2010 at 3:53 AM, Friedrich Romstedt
>> <friedrichromstedt at gmail.com> wrote:

>>> 2)  I noticed that the paver at some late point tried to switch from
>>> py2.5 to py2.6, what is rather strange to me.  I must have a look
>>> where precisely the build failed for this reason.  py2.6 is the
>>> DEFAULT_PYTHON_VERSION (iirc) in pavement.py, and changing it to 2.5
>>> fixed it.  Strange.
>>
>> Did you have the the bootstrap virtualenv set up and active? The following
>> should work from a clean checkout:
>> paver bootstrap
>> source bootstrap/bin/activate
>> python setupsconsegg.py install
>> paver dmg -p 2.5
>
> At that time my system Python was py2.5.  What I did is in
> http://friedrichromstedt.org/numpy/Releases/SystemSetup/2010.rst, and
> the offending call is logged in
> http://friedrichromstedt.org/numpy/Releases/Logs/10-10-15/paver-dmg.5.log.
>  I did not issue the commands you gave before calling $paver dmg.
> Also I installed paver using py2.5, and renamed the executable to
> paver2.5, to have different pavers for all the Pythons.  Is the -p 2.5
> switch fully equivalent?

2.5 is your default, so that's what was used to build the docs. Then
it tried to build an installer for 2.6, since that's the default given
in pavement.py. Nothing strange so far. The log ends with:
/bin/sh: /Library/Frameworks/Python.framework/Versions/2.6/bin/python:
No such file or directory
So you just didn't have python 2.6 installed, that's all.

The -p 2.5 is not for the paver version, but for the Python version to
compile numpy with.

>
> In pavement.py:
>
> @task
> @needs("pdf")
> @cmdopts([("python-version=", "p", "python version")])
> def dmg(options):
>    ...
>    _build_mpkg(pyver)
>
> def _build_mpkg(pyver):
>    ...
>    <calls setupegg.py bdist_mpkg>
>
> in setupegg.py:
> <calls just setup.py bdist_mpkg>
>
> So I thought it would create it locally, without using the /Framework
> installed numpy libs.  True?

True. The installed numpy was used only to build the docs. Basically,
during doc build numpy is imported and the docstrings are obtained
through introspection. Doc build finishes, you have a pdf. Then it
goes to part 2: build numpy against the /Framework installed Python.

>
> I would like to avoid $pave nuke, since I want to use the same repo
> directory for all Python version builds, and $nuke would remove the
> other installers.  They shouldn't interfere with each other since the
> Python version is encoded in the build directory.  The tasks clean and
> clean_bootstrap are okay with me.

It doesn't remove the installers. See the release.sh script in the
repo. It builds all installers at once.

>
> Seems that the docs show up with 1.4.1rc1 since I omitted the
> $setupsconsegg.py install, the autodoc found the previous install from
> the numpy-host/, where I didn't use 2.0.0.dev.

correct

>
> Seems that it surpisingly albeit my irgnorance of the instructions in
> pavement.py did work quite nicely.
>
> Can you explain in more detail what the virtualenv is for?  I guess
> it's to not install the numpy binaries built into the system's
> directory?

Yep. Numpy needs to be installed for the doc build to work, as I
explained above. You don't want to do that in global site-packages.

> Shouldn't there be a $deactivate call before cleaning the
> virtualenv?  Why cleaning the virtualenv at all?  I guess I might need
> to clean it for the different Pythons to not interfere, but doesn't
> virtualenv do the job too?

It's not cleaned. What is cleaned are the build/ and dist/ dirs. The
virtualenv is under bootstrap/, and the installer is under release/.
>
>> If that fails it's a bug. The first build is with python 2.6 if that's your
>> default python, it's needed to make sure the docs are built from the exact
>> same commit as the binary itself.
>
> "default Python" = DEFAULT_PYTHON in pavement.py?

I meant system default. And with first build I meant the one in release.sh.

>  I guess you mean
> the paver Python, the Python used to install Paver.  From the log
> http://friedrichromstedt.org/numpy/Releases/Logs/10-10-15/paver-dmg.5.log
> (the same as above), the docs were built, and there was no 2.6
> installed at that time.  It tried to call py2.6 *after* the successful
> call to pdflatex.

This should be clear now I think

>  I would like to know more about the internals of
> the build process, currently it's kind of "gray box" to me.
>
That's how it was for me too half a year ago. Maybe it still is a bit...

> What is DEFAULT_PYTHON for?  It is used in the options() call in
> pavement.py, why?  Else I see no substantial place of use in
> pavement.py.

It just defines what version you are building a dmg for when doing "$
paver dmg". Using the -p switch overrides that default.

>
>>> 3)  I found no v1.5.1 tag yet (yesterday).  Will the HEAD become 1.5.1?
>>
>> You mean the master branch? HEAD just points at the most recent commit on
>> your currently active branch. In that case no, 1.5.1 will be tagged from the
>> maintenance/1.5.x branch. The tag v1.5.1rc1 or v1.5.1 do not exist yet, tags
>> are only created once the actual release takes place. So tag v1.5.1rc1
>> should appear tomorrow.
>
> Okay, I didn't mention master since:
>
> 105osxpython:numpy-deployment Friedrich$ git branch
> * master
>
> This was like this from the beginning after $git clone
> http://github.com/numpy/numpy.git.  I thought owner-numpy
> project-numpy would be the official numpy repo, am I wrong?

You're not wrong. It's just git's behavior to only clone the master
branch by default. You can obtain other local branches as needed, for
example:
$ git checkout -b 1.5.x origin/maintenance/1.5.x

>
> Btw: how do I find out what commit a tag is pointing to, and how do I
> display those commit's details (using shell git commands)?
>
$ git log v1.5.0
commit c069eee07f2a976b3c2660670dc7bca6438ee94f
<...details...>

Cheers,
Ralf