Updating get_yt.sh, a question about defaults
Hi all, I've just issued a pull request to update the get_yt.sh script. https://bitbucket.org/yt_analysis/yt/pull-requests/1834/updating-miniconda-i... This uses miniconda to install yt into a user-writable prefix. The main advantage over the install script is that we don't need to bootstrap python. This is a big help on OS X 10.11, since the openssl headers are no longer available there. In the version of the script I just pull requested, I've set it to no longer build yt from source. Instead it does "conda install yt". I did this because it significantly simplifies the user-visible setup. In particular, users no longer need to have compilers installed to install everything, and we don't need to tell people to install XCode or a bunch of packages. It's also the case that the Conda versions of libraries are less useful for local development due to the way conda makes packages relocatable. My plan with this is to still keep the install script available, but offer the get_yt.sh script as an alternate install script more suitable for "quick" installations for users who aren't likely going to hack on yt. The install script will still be available, and that will produce a more hackable installation of yt. It also comes with more headaches, for example on 10.11 or newer users will be responsible for installing the OpenSSL headers using e.g. homebrew or macports. I know that in the past we've gone to great trouble to make sure people get hackable installations of yt out of the box. The cost of that is substantially increased pain of installation and also increased support workload for developers who respond to mailing list and IRC questions. I can easily set INST_YT_SOURCE=1 to be the default if the consensus is that people should always get hackable yt installations (e.g. source install with "setup.py develop" by default. What do you all think? -Nathan
A nice thing about installing from source, even for people who don't intend to do a bunch of development themselves, is that it makes it easier to get bugfix patches quickly. When people build from source, we can just say, "to get this fix into your yt, do so-and-so and re-run develop". Having nightly conda builds mitigates this concern somewhat, but that's still more latency, and it also makes it harder to point people to not-yet-accepted PRs to try out, as well. On Tue, Oct 27, 2015 at 2:42 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi all,
I've just issued a pull request to update the get_yt.sh script.
https://bitbucket.org/yt_analysis/yt/pull-requests/1834/updating-miniconda-i...
This uses miniconda to install yt into a user-writable prefix. The main advantage over the install script is that we don't need to bootstrap python. This is a big help on OS X 10.11, since the openssl headers are no longer available there.
In the version of the script I just pull requested, I've set it to no longer build yt from source. Instead it does "conda install yt". I did this because it significantly simplifies the user-visible setup. In particular, users no longer need to have compilers installed to install everything, and we don't need to tell people to install XCode or a bunch of packages. It's also the case that the Conda versions of libraries are less useful for local development due to the way conda makes packages relocatable.
My plan with this is to still keep the install script available, but offer the get_yt.sh script as an alternate install script more suitable for "quick" installations for users who aren't likely going to hack on yt. The install script will still be available, and that will produce a more hackable installation of yt. It also comes with more headaches, for example on 10.11 or newer users will be responsible for installing the OpenSSL headers using e.g. homebrew or macports.
I know that in the past we've gone to great trouble to make sure people get hackable installations of yt out of the box. The cost of that is substantially increased pain of installation and also increased support workload for developers who respond to mailing list and IRC questions. I can easily set INST_YT_SOURCE=1 to be the default if the consensus is that people should always get hackable yt installations (e.g. source install with "setup.py develop" by default.
What do you all think?
-Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
On Tue, Oct 27, 2015 at 5:04 PM, Andrew Myers <atmyers2@gmail.com> wrote:
A nice thing about installing from source, even for people who don't intend to do a bunch of development themselves, is that it makes it easier to get bugfix patches quickly. When people build from source, we can just say, "to get this fix into your yt, do so-and-so and re-run develop". Having nightly conda builds mitigates this concern somewhat, but that's still more latency, and it also makes it harder to point people to not-yet-accepted PRs to try out, as well.
One mitigation here is that we're trying to have bugfix releases out in a more timely fashion, which includes conda packages that users will get via this script. Speaking of, I need to do a release this week.... At the same time, users can pretty easily get source installs using get_yt.sh, they just need to change one line in the script. I could make a much louder comment about that to the user as well. I just think responding to a user saying "I want to install yt, let's try running this script" with "no! you must install xcode first" is not very friendly.
On Tue, Oct 27, 2015 at 2:42 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi all,
I've just issued a pull request to update the get_yt.sh script.
https://bitbucket.org/yt_analysis/yt/pull-requests/1834/updating-miniconda-i...
This uses miniconda to install yt into a user-writable prefix. The main advantage over the install script is that we don't need to bootstrap python. This is a big help on OS X 10.11, since the openssl headers are no longer available there.
In the version of the script I just pull requested, I've set it to no longer build yt from source. Instead it does "conda install yt". I did this because it significantly simplifies the user-visible setup. In particular, users no longer need to have compilers installed to install everything, and we don't need to tell people to install XCode or a bunch of packages. It's also the case that the Conda versions of libraries are less useful for local development due to the way conda makes packages relocatable.
My plan with this is to still keep the install script available, but offer the get_yt.sh script as an alternate install script more suitable for "quick" installations for users who aren't likely going to hack on yt. The install script will still be available, and that will produce a more hackable installation of yt. It also comes with more headaches, for example on 10.11 or newer users will be responsible for installing the OpenSSL headers using e.g. homebrew or macports.
I know that in the past we've gone to great trouble to make sure people get hackable installations of yt out of the box. The cost of that is substantially increased pain of installation and also increased support workload for developers who respond to mailing list and IRC questions. I can easily set INST_YT_SOURCE=1 to be the default if the consensus is that people should always get hackable yt installations (e.g. source install with "setup.py develop" by default.
What do you all think?
-Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
Unless I'm missing something, couldn't we get the best of both worlds by having the install_script install conda, install hg and yt through conda to make sure all of the dependencies are in place, and then use hg to pull yt from the repository and `python setup.py develop` there? That way, we'd avoid all of the openssl stuff that's been plaguing lots of systems, but we'd also still have yt dev tip and source available for people to hack on? that sort of setup is what i've been doing on systems manually where i cannot get things working with the main install script. Or perhaps I'm misinterpreting what you're doing now with the get_yt.sh script? Cameron On Tue, Oct 27, 2015 at 3:08 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
On Tue, Oct 27, 2015 at 5:04 PM, Andrew Myers <atmyers2@gmail.com> wrote:
A nice thing about installing from source, even for people who don't intend to do a bunch of development themselves, is that it makes it easier to get bugfix patches quickly. When people build from source, we can just say, "to get this fix into your yt, do so-and-so and re-run develop". Having nightly conda builds mitigates this concern somewhat, but that's still more latency, and it also makes it harder to point people to not-yet-accepted PRs to try out, as well.
One mitigation here is that we're trying to have bugfix releases out in a more timely fashion, which includes conda packages that users will get via this script. Speaking of, I need to do a release this week....
At the same time, users can pretty easily get source installs using get_yt.sh, they just need to change one line in the script. I could make a much louder comment about that to the user as well.
I just think responding to a user saying "I want to install yt, let's try running this script" with "no! you must install xcode first" is not very friendly.
On Tue, Oct 27, 2015 at 2:42 PM, Nathan Goldbaum <nathan12343@gmail.com> wrote:
Hi all,
I've just issued a pull request to update the get_yt.sh script.
https://bitbucket.org/yt_analysis/yt/pull-requests/1834/updating-miniconda-i...
This uses miniconda to install yt into a user-writable prefix. The main advantage over the install script is that we don't need to bootstrap python. This is a big help on OS X 10.11, since the openssl headers are no longer available there.
In the version of the script I just pull requested, I've set it to no longer build yt from source. Instead it does "conda install yt". I did this because it significantly simplifies the user-visible setup. In particular, users no longer need to have compilers installed to install everything, and we don't need to tell people to install XCode or a bunch of packages. It's also the case that the Conda versions of libraries are less useful for local development due to the way conda makes packages relocatable.
My plan with this is to still keep the install script available, but offer the get_yt.sh script as an alternate install script more suitable for "quick" installations for users who aren't likely going to hack on yt. The install script will still be available, and that will produce a more hackable installation of yt. It also comes with more headaches, for example on 10.11 or newer users will be responsible for installing the OpenSSL headers using e.g. homebrew or macports.
I know that in the past we've gone to great trouble to make sure people get hackable installations of yt out of the box. The cost of that is substantially increased pain of installation and also increased support workload for developers who respond to mailing list and IRC questions. I can easily set INST_YT_SOURCE=1 to be the default if the consensus is that people should always get hackable yt installations (e.g. source install with "setup.py develop" by default.
What do you all think?
-Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org
On Tuesday, October 27, 2015, Cameron Hummels <chummels@gmail.com> wrote:
Unless I'm missing something, couldn't we get the best of both worlds by having the install_script install conda, install hg and yt through conda to make sure all of the dependencies are in place, and then use hg to pull yt from the repository and `python setup.py develop` there? That way, we'd avoid all of the openssl stuff that's been plaguing lots of systems, but we'd also still have yt dev tip and source available for people to hack on? that sort of setup is what i've been doing on systems manually where i cannot get things working with the main install script. Or perhaps I'm misinterpreting what you're doing now with the get_yt.sh script?
Cameron
That's what get_yt.sh does if you tell it to build yt from source. The problem is that this requires having compilers available, which OSX and most Linux distros do not come with out of the box. Conda is also not so great for doing local development with, since there are issues with linking against C libraries provided by Conda on OSX.
On Tue, Oct 27, 2015 at 3:08 PM, Nathan Goldbaum <nathan12343@gmail.com <javascript:_e(%7B%7D,'cvml','nathan12343@gmail.com');>> wrote:
On Tue, Oct 27, 2015 at 5:04 PM, Andrew Myers <atmyers2@gmail.com <javascript:_e(%7B%7D,'cvml','atmyers2@gmail.com');>> wrote:
A nice thing about installing from source, even for people who don't intend to do a bunch of development themselves, is that it makes it easier to get bugfix patches quickly. When people build from source, we can just say, "to get this fix into your yt, do so-and-so and re-run develop". Having nightly conda builds mitigates this concern somewhat, but that's still more latency, and it also makes it harder to point people to not-yet-accepted PRs to try out, as well.
One mitigation here is that we're trying to have bugfix releases out in a more timely fashion, which includes conda packages that users will get via this script. Speaking of, I need to do a release this week....
At the same time, users can pretty easily get source installs using get_yt.sh, they just need to change one line in the script. I could make a much louder comment about that to the user as well.
I just think responding to a user saying "I want to install yt, let's try running this script" with "no! you must install xcode first" is not very friendly.
On Tue, Oct 27, 2015 at 2:42 PM, Nathan Goldbaum <nathan12343@gmail.com <javascript:_e(%7B%7D,'cvml','nathan12343@gmail.com');>> wrote:
Hi all,
I've just issued a pull request to update the get_yt.sh script.
https://bitbucket.org/yt_analysis/yt/pull-requests/1834/updating-miniconda-i...
This uses miniconda to install yt into a user-writable prefix. The main advantage over the install script is that we don't need to bootstrap python. This is a big help on OS X 10.11, since the openssl headers are no longer available there.
In the version of the script I just pull requested, I've set it to no longer build yt from source. Instead it does "conda install yt". I did this because it significantly simplifies the user-visible setup. In particular, users no longer need to have compilers installed to install everything, and we don't need to tell people to install XCode or a bunch of packages. It's also the case that the Conda versions of libraries are less useful for local development due to the way conda makes packages relocatable.
My plan with this is to still keep the install script available, but offer the get_yt.sh script as an alternate install script more suitable for "quick" installations for users who aren't likely going to hack on yt. The install script will still be available, and that will produce a more hackable installation of yt. It also comes with more headaches, for example on 10.11 or newer users will be responsible for installing the OpenSSL headers using e.g. homebrew or macports.
I know that in the past we've gone to great trouble to make sure people get hackable installations of yt out of the box. The cost of that is substantially increased pain of installation and also increased support workload for developers who respond to mailing list and IRC questions. I can easily set INST_YT_SOURCE=1 to be the default if the consensus is that people should always get hackable yt installations (e.g. source install with "setup.py develop" by default.
What do you all think?
-Nathan
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org <javascript:_e(%7B%7D,'cvml','yt-dev@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org <javascript:_e(%7B%7D,'cvml','yt-dev@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org <javascript:_e(%7B%7D,'cvml','yt-dev@lists.spacepope.org');> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Cameron Hummels NSF Postdoctoral Fellow Department of Astronomy California Institute of Technology http://chummels.org
participants (3)
-
Andrew Myers -
Cameron Hummels -
Nathan Goldbaum