SciPy.org Website Move and Redesign
At the scipy conference this year there were a number of people who expressed interest in moving the scipy.org website off of the current server and migrate away from using MoinMoinWiki. It turns out there have been a number of attempts at this over the past couple of years but nothing has ever stuck (as can be seen in the github org page). After discussing a number of approaches it sounds like the best strategy is one similar to what IPython and many other packages are using, namely using github pages and sphinx. The plan would be: 1) Mailing lists - Leave these as there's a lot of content there and we are worried about losing history and users if we even attempted to move this (plus we don't have a good alternative). 2) Scipy.org (and numpy.org) - Move the content to a repository under the Scipy organization on github that would contain a sphinx representation of the content that people can make pull requests to. There would be another repository that the built content would then be pushed to and that github would show as the website. David Warde-Farley has made an attempt at converting many of the scipy pages over to sphinx but it was never used (https://github.com/dwf/scipy-website). I noticed there were a few attempts at this a while ago (> 2 years). 3) Trac Projects - Numpy is in the process (or so we thought) of moving all the trac issues to github issues and as far as we are aware represents really the last reason these trac projects are still being used. Once this is done we can dispense with these as well. The scipy trac pages have not been moved at all and would also have to be done. These represent some effort but the website could make the transition before this is complete anyway. 4) Anything else? I am hoping that we can move the content quickly and set up the system first to get it off the ground and update the pages as we can. Hopefully we can get something going quickly and avoid the fates of previous attempts at this. Kyle
Kyle Mandli <kyle.mandli <at> gmail.com> writes: [clip]
1) Mailing lists - Leave these as there's a lot of content there and we are worried about losing history and users if we even attempted to move this (plus we don't have a good alternative).
Mailing lists are going to be needed. They don't necessarily need to be hosted on the current scipy.org machine, but it's probably best not to touch what has been working reliably.
2) Scipy.org (and numpy.org) - Move the content to a repository under the Scipy organization on github that would contain a sphinx representation of the content that people can make pull requests to. There would be another repository that the built content would then be pushed to and that github would show as the website. David Warde-Farley has made an attempt at converting many of the scipy pages over to sphinx but it was never used (https://github.com/dwf/scipy-website). I noticed there were a few attempts at this a while ago (> 2 years).
Note that the scipy.github.com site is up with a Git repo set up at https://github.com/scipy/scipy.org-new/ Similarly, numpy.scipy.org site has already moved to Github. I would see these repos as the central point for any further work. David's work is not merged there, but it could be merged (where applicable --- some of the content in scipy.org is outdated).
3) Trac Projects - Numpy is in the process (or so we thought) [clip]
I think the trac projects are a separate issue --- web site redesign can either precede or come after what is done with Trac. IMO, one of the first steps to do is to design a website layout skeleton that can be used appropriately on scipy.org, and daughter sites, docs.scipy.org, scipy-central.org, etc. This does not have to be in Sphinx format at first, just plain HTML+whatever, it's easy to hack in later. It also doesn't have to be great, but it does have to exist, and one has to think a bit about how and where to put the navigation tools. The amount of relevant "official" content on scipy.org is actually not very big, I think. The main issue is more on what to do with the Cookbook/Topical software sections, which in my opinion are not very well-suited to a Sphinx-generated static site. Why this hasn't moved forwards is probably largely due to not reaching a decision on what actually to do with this material. The current best bet as a replacement (apart from keeping these in Moin) would probably be scipy-central.org --- which I'm sure would also benefit greatly from additional love. Enthusiastic Django-capable people might be interested to take a look at improving it. -- Pauli Virtanen
On Wed, Jul 25, 2012 at 3:59 PM, Pauli Virtanen <pav@iki.fi> wrote:
Kyle Mandli <kyle.mandli <at> gmail.com> writes: [clip]
1) Mailing lists - Leave these as there's a lot of content there and we are worried about losing history and users if we even attempted to move this (plus we don't have a good alternative).
Mailing lists are going to be needed. They don't necessarily need to be hosted on the current scipy.org machine, but it's probably best not to touch what has been working reliably.
2) Scipy.org (and numpy.org) - Move the content to a repository under the Scipy organization on github that would contain a sphinx representation of the content that people can make pull requests to. There would be another repository that the built content would then be pushed to and that github would show as the website. David Warde-Farley has made an attempt at converting many of the scipy pages over to sphinx but it was never used (https://github.com/dwf/scipy-website). I noticed there were a few attempts at this a while ago (> 2 years).
Note that the scipy.github.com site is up with a Git repo set up at https://github.com/scipy/scipy.org-new/ Similarly, numpy.scipy.org site has already moved to Github.
I think the www.scipy.org site looks a lot nicer than the replacement, which still looks unfinished.
I would see these repos as the central point for any further work. David's work is not merged there, but it could be merged (where applicable --- some of the content in scipy.org is outdated).
3) Trac Projects - Numpy is in the process (or so we thought) [clip]
I think the trac projects are a separate issue --- web site redesign can either precede or come after what is done with Trac.
IMO, one of the first steps to do is to design a website layout skeleton that can be used appropriately on scipy.org, and daughter sites, docs.scipy.org, scipy-central.org, etc. This does not have to be in Sphinx format at first, just plain HTML+whatever, it's easy to hack in later. It also doesn't have to be great, but it does have to exist, and one has to think a bit about how and where to put the navigation tools.
The amount of relevant "official" content on scipy.org is actually not very big, I think.
The main issue is more on what to do with the Cookbook/Topical software sections, which in my opinion are not very well-suited to a Sphinx-generated static site. Why this hasn't moved forwards is probably largely due to not reaching a decision on what actually to do with this material.
The current best bet as a replacement (apart from keeping these in Moin) would probably be scipy-central.org --- which I'm sure would also benefit greatly from additional love. Enthusiastic Django-capable people might be interested to take a look at improving it.
Chuck
On Wed, Jul 25, 2012 at 7:36 PM, Charles R Harris <charlesr.harris@gmail.com
wrote:
On Wed, Jul 25, 2012 at 3:59 PM, Pauli Virtanen <pav@iki.fi> wrote:
Kyle Mandli <kyle.mandli <at> gmail.com> writes: [clip]
1) Mailing lists - Leave these as there's a lot of content there and we are worried about losing history and users if we even attempted to move this (plus we don't have a good alternative).
Mailing lists are going to be needed. They don't necessarily need to be hosted on the current scipy.org machine, but it's probably best not to touch what has been working reliably.
2) Scipy.org (and numpy.org) - Move the content to a repository under the Scipy organization on github that would contain a sphinx representation of the content that people can make pull requests to. There would be another repository that the built content would then be pushed to and that github would show as the website. David Warde-Farley has made an attempt at converting many of the scipy pages over to sphinx but it was never used (https://github.com/dwf/scipy-website). I noticed there were a few attempts at this a while ago (> 2 years).
Note that the scipy.github.com site is up with a Git repo set up at https://github.com/scipy/scipy.org-new/ Similarly, numpy.scipy.org site has already moved to Github.
I think the www.scipy.org site looks a lot nicer than the replacement, which still looks unfinished.
I disagree and am in favor of the replacement though I probably don't have the spare cycles to contribute in any meaningful way. ~A.
I would see these repos as the central point for any further work. David's work is not merged there, but it could be merged (where applicable --- some of the content in scipy.org is outdated).
3) Trac Projects - Numpy is in the process (or so we thought) [clip]
I think the trac projects are a separate issue --- web site redesign can either precede or come after what is done with Trac.
IMO, one of the first steps to do is to design a website layout skeleton that can be used appropriately on scipy.org, and daughter sites, docs.scipy.org, scipy-central.org, etc. This does not have to be in Sphinx format at first, just plain HTML+whatever, it's easy to hack in later. It also doesn't have to be great, but it does have to exist, and one has to think a bit about how and where to put the navigation tools.
The amount of relevant "official" content on scipy.org is actually not very big, I think.
The main issue is more on what to do with the Cookbook/Topical software sections, which in my opinion are not very well-suited to a Sphinx-generated static site. Why this hasn't moved forwards is probably largely due to not reaching a decision on what actually to do with this material.
The current best bet as a replacement (apart from keeping these in Moin) would probably be scipy-central.org --- which I'm sure would also benefit greatly from additional love. Enthusiastic Django-capable people might be interested to take a look at improving it.
Chuck
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Wed, Jul 25, 2012 at 6:57 PM, Anthony Scopatz <scopatz@gmail.com> wrote:
I think the www.scipy.org site looks a lot nicer than the replacement, which still looks unfinished.
I disagree and am in favor of the replacement though I probably don't have the spare cycles to contribute in any meaningful way.
I think the point that's more important here is that the new approach is *maintainable* moving forward and makes it trivial to delegate website work to a separate team of contributors. The moin-based setup is more or less impossible to update in any meaningful way, so for all intents and purposes it's a complete dead-end. Cheers, f
Charles R Harris <charlesr.harris <at> gmail.com> writes: [clip]
I think the www.scipy.org site looks a lot nicer than the replacement, which still looks unfinished.
Well, it *is* unfinished, and the point in this thread was to make it finished. The current state is not very relevant for this discussion. Pauli
On Thu, Jul 26, 2012 at 3:59 PM, Pauli Virtanen <pav@iki.fi> wrote:
Charles R Harris <charlesr.harris <at> gmail.com> writes: [clip]
I think the www.scipy.org site looks a lot nicer than the replacement, which still looks unfinished.
Well, it *is* unfinished, and the point in this thread was to make it finished. The current state is not very relevant for this discussion.
May I suggest making it look more like the www.scipy.org site then? I think that site looks pretty good. Chuck
On Fri, Jul 27, 2012 at 12:21 AM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Thu, Jul 26, 2012 at 3:59 PM, Pauli Virtanen <pav@iki.fi> wrote:
Charles R Harris <charlesr.harris <at> gmail.com> writes: [clip]
I think the www.scipy.org site looks a lot nicer than the replacement, which still looks unfinished.
Well, it *is* unfinished, and the point in this thread was to make it finished. The current state is not very relevant for this discussion.
May I suggest making it look more like the www.scipy.org site then? I think that site looks pretty good.
Well, I'd be interested to see what someone with some design talent could do. scipy.org doesn't look that good imho. And can we please lose the baby blue background? The days of hand-written html with plain colored backgrounds are over. A more interesting discussion is the content. scipy.org talks about the *project* scipy, while http://scipy.github.com/ is about the *ecosystem*. I'd prefer the front page to be about the ecosystem, and have a new page about the project. Ralf
On Thu, Jul 26, 2012 at 11:17 PM, Ralf Gommers <ralf.gommers@googlemail.com> wrote:
Well, I'd be interested to see what someone with some design talent could do. scipy.org doesn't look that good imho. And can we please lose the baby blue background? The days of hand-written html with plain colored backgrounds are over.
A more interesting discussion is the content. scipy.org talks about the *project* scipy, while http://scipy.github.com/ is about the *ecosystem*. I'd prefer the front page to be about the ecosystem, and have a new page about the project.
And precisely having the site moved over to a system where: - it's trivial to delegate website work to a separate team who can collaborate with the regular github flow (branches, PRs, etc) to manage the web content. - form and content are separated, with templates and CSS for all formal display makes progress on both of those fronts possible. The ipython site isn't about to win any web design awards, and we deliberately kept it very minimalistic, but it shows that it's easy to make a sphinx site that looks very different from your typical sphinx doc build. With a bit more work, a talented web team could make it look even more different, while still using a toolchain (sphinx, reST and github) that we use for other things and therefore leveraging existing skills. Cheers, f
On Fri, Jul 27, 2012 at 11:35 AM, Fernando Perez <fperez.net@gmail.com> wrote:
And precisely having the site moved over to a system where:
- it's trivial to delegate website work to a separate team who can collaborate with the regular github flow (branches, PRs, etc) to manage the web content.
- form and content are separated, with templates and CSS for all formal display
Is this transition still on track? We should push it through, even if it means living with a slightly outdated website for a week or two. (I'm sure we'll all feel much more comfortable updating the web site using PRs than whatever the current mechanism might be.) Stéfan
Hey Kyle and Pauli, Ognen is out this week, but he will be helpful for how to best work with on server options. We'll be glad to help getting the scipy server set up as needed to host static/Django/whatever. I am, unfortunately, heading out on vacation early tomorrow morning and am off the grid through next week. I'll catch up with the conversation at that point. eric On Jul 25, 2012, at 4:59 PM, Pauli Virtanen wrote:
Kyle Mandli <kyle.mandli <at> gmail.com> writes: [clip]
1) Mailing lists - Leave these as there's a lot of content there and we are worried about losing history and users if we even attempted to move this (plus we don't have a good alternative).
Mailing lists are going to be needed. They don't necessarily need to be hosted on the current scipy.org machine, but it's probably best not to touch what has been working reliably.
2) Scipy.org (and numpy.org) - Move the content to a repository under the Scipy organization on github that would contain a sphinx representation of the content that people can make pull requests to. There would be another repository that the built content would then be pushed to and that github would show as the website. David Warde-Farley has made an attempt at converting many of the scipy pages over to sphinx but it was never used (https://github.com/dwf/scipy-website). I noticed there were a few attempts at this a while ago (> 2 years).
Note that the scipy.github.com site is up with a Git repo set up at https://github.com/scipy/scipy.org-new/ Similarly, numpy.scipy.org site has already moved to Github.
I would see these repos as the central point for any further work. David's work is not merged there, but it could be merged (where applicable --- some of the content in scipy.org is outdated).
3) Trac Projects - Numpy is in the process (or so we thought) [clip]
I think the trac projects are a separate issue --- web site redesign can either precede or come after what is done with Trac.
IMO, one of the first steps to do is to design a website layout skeleton that can be used appropriately on scipy.org, and daughter sites, docs.scipy.org, scipy-central.org, etc. This does not have to be in Sphinx format at first, just plain HTML+whatever, it's easy to hack in later. It also doesn't have to be great, but it does have to exist, and one has to think a bit about how and where to put the navigation tools.
The amount of relevant "official" content on scipy.org is actually not very big, I think.
The main issue is more on what to do with the Cookbook/Topical software sections, which in my opinion are not very well-suited to a Sphinx-generated static site. Why this hasn't moved forwards is probably largely due to not reaching a decision on what actually to do with this material.
The current best bet as a replacement (apart from keeping these in Moin) would probably be scipy-central.org --- which I'm sure would also benefit greatly from additional love. Enthusiastic Django-capable people might be interested to take a look at improving it.
-- Pauli Virtanen
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Wed, Jul 25, 2012 at 5:59 PM, Pauli Virtanen <pav@iki.fi> wrote:
Kyle Mandli <kyle.mandli <at> gmail.com> writes: [clip]
1) Mailing lists - Leave these as there's a lot of content there and we are worried about losing history and users if we even attempted to move this (plus we don't have a good alternative).
Mailing lists are going to be needed. They don't necessarily need to be hosted on the current scipy.org machine, but it's probably best not to touch what has been working reliably.
2) Scipy.org (and numpy.org) - Move the content to a repository under the Scipy organization on github that would contain a sphinx representation of the content that people can make pull requests to. There would be another repository that the built content would then be pushed to and that github would show as the website. David Warde-Farley has made an attempt at converting many of the scipy pages over to sphinx but it was never used (https://github.com/dwf/scipy-website). I noticed there were a few attempts at this a while ago (> 2 years).
Note that the scipy.github.com site is up with a Git repo set up at https://github.com/scipy/scipy.org-new/ Similarly, numpy.scipy.org site has already moved to Github.
I would see these repos as the central point for any further work. David's work is not merged there, but it could be merged (where applicable --- some of the content in scipy.org is outdated).
3) Trac Projects - Numpy is in the process (or so we thought) [clip]
I think the trac projects are a separate issue --- web site redesign can either precede or come after what is done with Trac.
IMO, one of the first steps to do is to design a website layout skeleton that can be used appropriately on scipy.org, and daughter sites, docs.scipy.org, scipy-central.org, etc. This does not have to be in Sphinx format at first, just plain HTML+whatever, it's easy to hack in later. It also doesn't have to be great, but it does have to exist, and one has to think a bit about how and where to put the navigation tools.
The amount of relevant "official" content on scipy.org is actually not very big, I think.
The main issue is more on what to do with the Cookbook/Topical software sections, which in my opinion are not very well-suited to a Sphinx-generated static site. Why this hasn't moved forwards is probably largely due to not reaching a decision on what actually to do with this material.
The current best bet as a replacement (apart from keeping these in Moin) would probably be scipy-central.org --- which I'm sure would also benefit greatly from additional love. Enthusiastic Django-capable people might be interested to take a look at improving it.
-- Pauli Virtanen
Hi everyone, the scipy-central.org "maintainer" here; in inverted commas, because I've been doing no maintenance over the past 12 to 18 months. I'd like to offer all the scipy-central.org material over to the community to keep going with it and improve it. I expected to have time to do it myself, but new jobs and a different career path have prevented me from implementing improvements that Pauli and several others have proposed in the past. This might be a good time to consider the issue, even just from a branding perspective, to get it all looking coherent. If anyone wants to get back to me on this, I can pass over 1. Some/all of the domain names (those not transferred I will let lapse on the next renewal in June 2013) scipy-central.org(DNS) scpyce.org(DNS) scipy-central.com(MX) Forward ( http://scipy-central.org ) scipy-central.net(MX) Forward ( http://scipy-central.org ) scipycentral.com(MX) Forward ( http://scipy-central.org ) scipycentral.net(MX) Forward ( http://scipy-central.org ) scipycentral.org(MX) Forward ( http://scipy-central.org ) scpyce.com(MX) Forward ( http://scpyce.org ) scpyce.net(MX) Forward ( http://scpyce.org ) 2. all Django code, media files, templates, CSS and HTML, including detailed instructions on how to install and use it are already at https://github.com/kgdunn/SciPyCentral Please take a look at this to judge the complexity of the move. 3. backup scripts: local and off-site 4. the PostgreSQL database [23Mb] 5. monthly backup snapshots going back to July 2011, when the site started [450Mb] 6. the Google Analytics account (not sure how to transfer this) 7. the original artwork files for the media There are some interesting page hit and search analytics in raw data (item 5) and from Google (item 6) that someone might want to look at. I currently host the site on Webfaction. The new owner(s) will require a similar unix-type server with: * Python and Django, * webserver (e.g. Apache, or nginx), * ability to email for account creation, password resets and submission confirmation * rsync for remote backup, * and other miscellaneous utilities: South, Pygments, Sphinx, Haystack search with Xapian, PIL, etc The new owner(s) will also inherit 2 pull requests and 45 issues on the above-mentioned Github site. Other proposals made in the Scipy-Dev mailing lists over the past year also exist. I'm in no rush to hand it over; just putting it out there that someone or some group more capable and with more time can definitely do a better job of it than me. Let me know how the community wants to proceed: please let's keep the discussion public so everyone can contribute. Kevin
participants (9)
-
Anthony Scopatz -
Charles R Harris -
Eric Jones -
Fernando Perez -
Kevin Dunn -
Kyle Mandli -
Pauli Virtanen -
Ralf Gommers -
Stéfan van der Walt