As most of you know, I'm at an academic institution and have a strong belief that open source should mesh better with the academic world then it currently does. One of the problems, is that it is not easy to just point to open source software as a "publication" which is one of the areas that most tenure-granting boards look at when deciding on a candidate. To help with that a little-bit, I'd really like to get a peer-reviewed on-line journal for code contributed to SciPy started. Not to implicate him if he no longer thinks it's a good idea, but I first started thinking seriously about this at SciPy 2006 when Fernando Perez mentioned he had seen this model with a math-related software package and thought we could try something similar. Exactly what form that will take is worth some discussion. Most "publications" are viewed highly because they are "peer-reviewed". It seems to me, that we have all the makings of a community that could peer-review code contributed to SciPy. I'd like to see two parts to "the journal" (maybe they are separate). One is the code itself. Not all of the code currently in SciPy is "journal" quality (I look no further than many of my own contributions...) We would need a way to specify what is "published" SciPy and what is not (besides the sandbox mechansim). The other aspect to the SciPy Journal is an "article" that goes into detail about some particular code contributed to SciPy. Not all code will need this, but many contributions will. This is where the design decisions are discussed and recorded. Such a project needs many associate editors who would help find reviewers and make decisions about publication. If this project works, it is one means that would allow me to spend more time on SciPy and "have it count" toward academic rank and status. I have not thought it through well enough to have the details ironed out, but I'm soliciting feedback from you who would likely be involved. Besides your comments, I would like a response on at least two questions: +1, -1, +0, -0 voting. 1) Do you like the idea ? 2) Could you help (i.e. as an editor, reviewer, etc.)?
Travis Oliphant wrote:
As most of you know, I'm at an academic institution and have a strong belief that open source should mesh better with the academic world then it currently does. One of the problems, is that it is not easy to just point to open source software as a "publication" which is one of the areas that most tenure-granting boards look at when deciding on a candidate.
To help with that a little-bit, I'd really like to get a peer-reviewed on-line journal for code contributed to SciPy started. Not to implicate him if he no longer thinks it's a good idea, but I first started thinking seriously about this at SciPy 2006 when Fernando Perez mentioned he had seen this model with a math-related software package and thought we could try something similar.
Indeed! Another example is the _Journal of Statistical Software_ which is a free, online, peer-reviewed journal focused on statistical software. Looking at the articles, I sometimes wonder if it just should have been called the _Journal of R_, but I digress. It seems to be well-done.
1) Do you like the idea ?
+1!
2) Could you help (i.e. as an editor, reviewer, etc.)?
+1 -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On 9/30/06, Travis Oliphant <oliphant@ee.byu.edu> wrote:
Besides your comments, I would like a response on at least two questions: +1, -1, +0, -0 voting.
1) Do you like the idea ?
+1 I like the idea, but at least in the world of computer graphics research, writing about the implementation of some software is not usually considered a topic for a publication. At least I don't know of any mainstream journals or conferences that have this kind of slant. Journal of Graphics Tools is probably the closest, but those papers usually present a clever new algorithm of some sort, or useful twist on an old one. Novelty is always one of the main criteria for reviewing anywhere I've sent papers. So something like "implementing an FFT module for SciPy" wouldn't go over well with any of the venues I'm familiar with. "There's nothing novel about the FFT" would be the rejection. The idea is pretty devious though.. I like it. Excellent software infrastructure really does lead to increased scientific progress, and it doesn't happen by itself. So a way for academics to get the kudos they need to continue such efforts would be great.
2) Could you help (i.e. as an editor, reviewer, etc.)?
+1 --bb
On Sep 30, 2006, at 3:42 AM, Bill Baxter wrote:
Novelty is always one of the main criteria for reviewing anywhere I've sent papers. So something like "implementing an FFT module for SciPy" wouldn't go over well with any of the venues I'm familiar with. "There's nothing novel about the FFT" would be the rejection.
Which is a shame, because I've encountered several instances lately of "nothing novel algorithms" that don't work as published. When you finally dig up somebody's code that "works", you discover that there's some little hack needed, or just some branch of the algorithm that they forgot to publish. This isn't unique to computational science, of course; the experimental world is full of results that can only be obtained by their "discoverers" (code fusion, anybody?); but I think it's particular inexcusable in the computational realm, since it's entirely feasible to hand your entire "experimental apparatus" to anybody that wants it. I think it would be a wonderful thing for the reproducibility of scientific results if authors were required to publish their code and their data instead of just a description of their code and plots of their data. Nonetheless, I understand there needs to be some sort of distinction between "yet another implementation of a very well understood and accepted algorithm" (FFT is arguably in this category) and "here's a way to do something novel along with how I actually did it". Taking a bunch of accepted algorithms and putting them together in a way that they can talk to each other and are reliable and the whole thing makes sense *should* be interesting to the scientific community.
Travis Oliphant wrote:
As most of you know, I'm at an academic institution and have a strong belief that open source should mesh better with the academic world then it currently does. One of the problems, is that it is not easy to just point to open source software as a "publication" which is one of the areas that most tenure-granting boards look at when deciding on a candidate.
To help with that a little-bit, I'd really like to get a peer-reviewed on-line journal for code contributed to SciPy started. Not to implicate him if he no longer thinks it's a good idea, but I first started thinking seriously about this at SciPy 2006 when Fernando Perez mentioned he had seen this model with a math-related software package and thought we could try something similar.
Exactly what form that will take is worth some discussion. Most "publications" are viewed highly because they are "peer-reviewed". It seems to me, that we have all the makings of a community that could peer-review code contributed to SciPy. I'd like to see two parts to "the journal" (maybe they are separate). One is the code itself. Not all of the code currently in SciPy is "journal" quality (I look no further than many of my own contributions...)
I like the peer-review idea for contributing code. I am kind of new in the academic research environment (just started my PhD a few months ago), so I am not very familiar with some of the constraints of a journal (that is quality requirement, expectation on the scientific level, etc...). Reading other responses to this email, I will take a look at mentioned publications. So as much as I am not 'experienced enough' on the whole academic publication thing, I like it.
1) Do you like the idea ?
+1
2) Could you help (i.e. as an editor, reviewer, etc.)?
+1 if I can help (What would be the requirements to be a reviewer, etc... ?) David
Travis Oliphant wrote:
...
Besides your comments, I would like a response on at least two questions: +1, -1, +0, -0 voting.
1) Do you like the idea ? +1
2) Could you help (i.e. as an editor, reviewer, etc.)? +1
-- Steven H. Rogers, Ph.D., steve@shrogers.com Weblog: http://shrogers.com/weblog "He who refuses to do arithmetic is doomed to talk nonsense." -- John McCarthy
Besides your comments, I would like a response on at least two questions: +1, -1, +0, -0 voting.
1) Do you like the idea ?
+1
2) Could you help (i.e. as an editor, reviewer, etc.)?
+1 Those of you in the C++ community might be familiar with boost (www.boost.org) a peer-reviewed set of STL-like libraries that has gained a great following in the C++ community. A large part of their review process is focused on the API -- is it STL-like, does it make sense, will it grow, etc. A lot of scipy areas are loose wrappers around existing fortran code, and so have APIs that are somewhat un-pythoninc. It would be nice if part of the review process for libraries / publications could involve API feedback as well. ...Eric
Eric Jonas wrote:
Those of you in the C++ community might be familiar with boost (www.boost.org) a peer-reviewed set of STL-like libraries that has gained a great following in the C++ community. A large part of their review process is focused on the API -- is it STL-like, does it make sense, will it grow, etc. A lot of scipy areas are loose wrappers around existing fortran code, and so have APIs that are somewhat un-pythoninc. It would be nice if part of the review process for libraries / publications could involve API feedback as well.
The example that I believe Fernando was referring to is the GAP Council which, formally and informally, reviews contributions to the GAP computational group theory software. http://www-gap.mcs.st-and.ac.uk/Contacts/People/Council/council.html http://www-gap.mcs.st-and.ac.uk/Contacts/submit.html Both of us heard a talk by Steve Linton at the SAGE Days 2006 workshop where he talked about this. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
"Travis" == Travis Oliphant <oliphant@ee.byu.edu> writes:
Travis> 1) Do you like the idea ? I think this is a very good idea. Some people are likely to claim "nothing new in the particular implementation". However, you could insist that code contributed here should be unit tested and reasonably integration tested (how often do we see anything close to that in other papers?). It might be a good idea to have two kinds of contributions, "original/new ideas/algorithms/applications" and "new implementations". Most papers, it would appear, are read by a handful and used by an even smaller subset of the audience. The advantage of something contributed to SciPy is that it has (on the average) far greater utility to the whole community. Besides, I think it is reasonable to argue that good code usually takes as long as (if not longer than) writing a paper. Travis> 2) Could you help (i.e. as an editor, reviewer, etc.)? I can try to help with this. cheers, prabhu
Prabhu Ramachandran wrote:
"Travis" == Travis Oliphant <oliphant@ee.byu.edu> writes:
Travis> 1) Do you like the idea ?
I think this is a very good idea. Some people are likely to claim "nothing new in the particular implementation". However, you could insist that code contributed here should be unit tested and reasonably integration tested (how often do we see anything close to that in other papers?). It might be a good idea to have two kinds of contributions, "original/new ideas/algorithms/applications" and "new implementations". Most papers, it would appear, are read by a handful and used by an even smaller subset of the audience. The advantage of something contributed to SciPy is that it has (on the average) far greater utility to the whole community. Besides, I think it is reasonable to argue that good code usually takes as long as (if not longer than) writing a paper.
For comparison, here are excerpts from the JStatSoft submission instructions. http://www.jstatsoft.org/instructions.php """JSS will publish 1. Manuals, user's guides, and other forms of description of statistical software, together with the actual software in human-readable form (peer-reviewed) 2. Code snippets -- small code projects, any language (section editors Hornik and Koenker, peer-reviewed). 3. Special issues on topics in statistical computing (guest editors, peer-reviewed, by invitation only, suggestions welcome). 4. A yearly special issue documenting progress of major statistical software projects (section editor Rossini, by invitation only, suggestions welcome) . 5. Reviews of Books on statistical computing and software. (section editor Gentleman, by invitation only, suggestions welcome) . 6. Reviews and comparisons of statistical software (section editors Unwin and Hartman, by invitation only, suggestions welcome). The typical JSS paper will have a section explaining the statistical technique, a section explaining the code, a section with the actual code, and a section with examples. All sections will be made browsable as well as downloadable. The papers and code should be accessible to a broad community of practitioners, teachers, and researchers in the field of statistics. """ However: """If code does something standard (for instance compute an incomplete beta in Fortran) it is only acceptable if it is better than the alternatives. On the other hand, if it does an incomplete non-central beta in Xlisp-Stat, then it merely has to show that it works well. """ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
For comparison, here are excerpts from the JStatSoft submission instructions.
http://www.jstatsoft.org/instructions.php
"""JSS will publish
1. Manuals, user's guides, and other forms of description of statistical software, together with the actual software in human-readable form (peer-reviewed) 2. Code snippets -- small code projects, any language (section editors Hornik and Koenker, peer-reviewed). 3. Special issues on topics in statistical computing (guest editors, peer-reviewed, by invitation only, suggestions welcome). 4. A yearly special issue documenting progress of major statistical software projects (section editor Rossini, by invitation only, suggestions welcome) . 5. Reviews of Books on statistical computing and software. (section editor Gentleman, by invitation only, suggestions welcome) . 6. Reviews and comparisons of statistical software (section editors Unwin and Hartman, by invitation only, suggestions welcome).
The typical JSS paper will have a section explaining the statistical technique, a section explaining the code, a section with the actual code, and a section with examples. All sections will be made browsable as well as downloadable. The papers and code should be accessible to a broad community of practitioners, teachers, and researchers in the field of statistics. """
However:
"""If code does something standard (for instance compute an incomplete beta in Fortran) it is only acceptable if it is better than the alternatives. On the other hand, if it does an incomplete non-central beta in Xlisp-Stat, then it merely has to show that it works well. """
I think these are good ideas that we can glean from. It is really nice that www.jstatsoft.org is already out there. There is enough interest that we should try to push this forward. I like the idea of volumes tracked by year and issues (or numbers) simply tracked by article. Let's go with that. We need to come up with guidelines for what can be published. There is a lot to discuss here. We don't have to nail it down exactly, but we should have a general idea of what we consider "publishable". Given the existence of other journals that are more "general-purpose," I'd really like to have a journal that focuses on code written for Python. Is that feasible? I don't see why not. I think that Python makes an excellent language to express scientific algorithms in. I think it should be a requirement that the contribution have some relevance to computing with Python. I definitely think there will be different "tiers" of contributions. These should be recognized as such by the existence of two "types of contributions". My preferred approach is to have this divided into two Journals (call them A and B for now) which clarify the difference in novelty. The "A" journal would discuss contributions where a novel algorithm, implementation, or idea is expressed while the "B" contributions are module-style Python implementations of previously-known algorithms. Another candidate for the "A" journal might be documentation and "packaging" of several "B" contributions into something that could be a SciPy sub-package. I could see a typical publication needing two parts code and write-up. Code: 1) working code 2) unit-tests 3) docstring-style docs Write-up 1) Description of how the algorithm fits into the world at a "reasonably" high level. 2) Discussion of how the algorithm was implemented and the design decisions behind the interface, some references to other implementations would also be useful here (whether in Python or not). 3) Technical description. I think there are some "ready-made" publications already sitting in the SciPy SVN tree. Getting the raw code "published" would be the "review" process that Robert is pushing for and could be under-taken by people who are not the original author. I think we should tag code that has been "published" in some way in the SciPy tree so that 1) It is searchable (remember the discussion about adding code-category-tags to SciPy). 2) You can know whether or not an algorithm has had that review. Just to clarify my views, I don't think we need to make it a requirement that the code "go in to SciPy" to be published. In other-words, people shouldn't feel like they have to "give-up" their code. I only want to require that it works with Python. Other modules that are separately installed could still be "published" as long as the code was provided. What should the title be? How about Journal of Scientific Computing with Python for boring??? Anxious to hear your ideas. My time-frame for this is that I'd like to get an issue out by January of next year (remember an issue is just a single publication). -Travis
On 10/2/06, Travis Oliphant <oliphant.travis@ieee.org> wrote:
Given the existence of other journals that are more "general-purpose," I'd really like to have a journal that focuses on code written for Python. Is that feasible? I don't see why not. I think that Python makes an excellent language to express scientific algorithms in. I think it should be a requirement that the contribution have some relevance to computing with Python.
Even if it is all Python for the time being, I think I would prefer a more generic sounding name, like "Journal of Scientific Computing in High Level Languages". Ok, that sucks, but still, programming langauges come and go, and staking the journal's name to a particular one seems to be saying the ideas don't matter as much as the technology used. Sounds maybe more like a Ziff-Davis magazine than a Journal. --bb
Bill Baxter wrote:
On 10/2/06, Travis Oliphant <oliphant.travis@ieee.org> wrote:
Given the existence of other journals that are more "general-purpose," I'd really like to have a journal that focuses on code written for Python. Is that feasible? I don't see why not. I think that Python makes an excellent language to express scientific algorithms in. I think it should be a requirement that the contribution have some relevance to computing with Python.
Even if it is all Python for the time being, I think I would prefer a more generic sounding name, like "Journal of Scientific Computing in High Level Languages".
I have mixed feelings here. I, too, am concerned that a journal which expounds scientific computing with Python might be seen as too "Ziff-Davis" like (although it will certainly have a much different focus than their magazines). On the other hand, there are already a lot of "general-purpose" journals for publishing algorithms in scientific computing. We need a journal that documents the development of tools for an open source platform for conducting scientific computing. Right now that platform for us is Python. Should it change, then another journal gets started. I think this is an example of where you lose benefit by trying to be too general. I don't think the reputation of the journal will be altered (in the long run) by whether or not the word Python is in the name. In fact, I'd like to emphasize the use of Python for scientific code development in place of other choices which are also available. So, perhaps I'm not "ambitious" enough to create a publication that outlasts Python, but in my mind, the journal lives and dies by the use of Python in scientific computing. There are just too many other journals that take the "all languages" welcome view point and provide you with papers that describe algorithms but are difficult to reproduce. I think we should make it clear that this journal details how Python can be used for computing (and for explaining algorithms). -Travis
On Mon, 02 Oct 2006, Travis Oliphant apparently wrote:
We need a journal that documents the development of tools for an open source platform for conducting scientific computing. Right now that platform for us is Python.
A journal has a mission statement where this can be specified. It does not need to be in its title. And the title needs not to turn away people who would e.g., write C extensions or create something like f2py. More titles (any puns intended): Objective Science Innovations in Scientific Computing Extending Scientific Computing Code Fusion (serendipity; but I forget whom) Applied Scientific Computing Scientific Computing Insight But to be specific it would be: SciPy Journal Oh well, Alan Isaac
Alan G Isaac wrote:
On Mon, 02 Oct 2006, Travis Oliphant apparently wrote:
We need a journal that documents the development of tools for an open source platform for conducting scientific computing. Right now that platform for us is Python.
A journal has a mission statement where this can be specified. It does not need to be in its title.
And the title needs not to turn away people who would e.g., write C extensions or create something like f2py.
More titles (any puns intended): Objective Science Innovations in Scientific Computing Extending Scientific Computing Code Fusion (serendipity; but I forget whom) Applied Scientific Computing Scientific Computing Insight
Good name suggestions. I like the names Applied Scientific Computing Applied Computational Science -Travis
On Mon, 02 Oct 2006, Travis Oliphant apparently wrote:
I like the names Applied Scientific Computing Applied Computational Science
I think the first is more accurate? The journal will also have to be hosted someplace reasonably stable, so that access to past issues is sustained. Might Enthought be willing to provide space for this? (I have absolutely no right to ask this and no clue about its appropriateness; I am just brain storming.) Assuming LaTeX only submissions, good style file is important right off the bat. I like the JSS style. This is Copyright (C) 2004 Achim Zeileis Achim.Zeileis@wu-wien.ac.at If others like it too, I'm willing to ask him for permission to hack it. Cheers, Alan Isaac
On Mon, Oct 02, 2006 at 10:56:01PM -0400, Alan G Isaac wrote:
Assuming LaTeX only submissions, good style file is important right off the bat. I like the JSS style. This is Copyright (C) 2004 Achim Zeileis Achim.Zeileis@wu-wien.ac.at If others like it too, I'm willing to ask him for permission to hack it.
I guess we are getting ahead of ourselves, but since the topic was brough up: The IEEE Journal layout is distributed under the Perl Artistic License at http://www.ctan.org/tex-archive/macros/latex/contrib/IEEEtran/ with the additional advantage that it has a LyX template (included in the LyX distribution). The idea of a journal for pythonic science appeals to me as a student. In order to complete theses, large pieces of code are often written; yet, for all that time spent doing that there is no reward in terms of a publication. I agree with David that this may motivate contributing scientists to write better code. We could probably do with a review of the level of testing done in scipy/numpy ourselves ;) Regards Stéfan
On 10/3/06, Travis Oliphant <oliphant@ee.byu.edu> wrote:
Bill Baxter wrote:
On 10/2/06, Travis Oliphant <oliphant.travis@ieee.org> wrote:
Even if it is all Python for the time being, I think I would prefer a more generic sounding name, like "Journal of Scientific Computing in High Level Languages".
We need a journal that documents the development of tools for an open source platform for conducting scientific computing. Right now that platform for us is Python.
So, perhaps I'm not "ambitious" enough to create a publication that outlasts Python
Well I think it's pretty ambitious to restrict the potential pool of submitters to just one language from the outset. Maybe the policies can worm around that a little. LIke "ok we call it the Journal of Scientific Python, but really we'll accept things in other languages too, as long as they are really good and take advantage of the same sorts of features that make Python great." So that way if someone goes and creates SciRuby it could be published in the Journal of Scientific Python, too. ( But after that they may want to go and create the Journal of Scientific Ruby.) In that way it could serve as a way to get ideas for features that are must-haves for the next generation python. -- There could be a subtitle that usually no-one ever mentions but the official title is like: THE JOURNAL OF SCIENTIFIC PYTHON and science with dynamic programming languages -- Another side-benefit of it being very obviously Python-centric is that it becomes a very visible sign of the size, vitality and value of the community of scientific python users, which could help lend weight to requests for language changes. Being able to create web pages better and faster is great, but if your programming language is actually saving lives by analyzing CAT scan data or something like that, then that's really something to be proud of. And if the makers of that software say they could do a better job with feature X that seems pretty compelling. --bb
On Oct 2, 2006, at 3:22 AM, Travis Oliphant wrote:
Just to clarify my views, I don't think we need to make it a requirement that the code "go in to SciPy" to be published. In other-words, people shouldn't feel like they have to "give-up" their code. I only want to require that it works with Python. Other modules that are separately installed could still be "published" as long as the code was provided.
With this clarification, definitely count me as +1 on this scheme. I was supportive already, but was hesitant that a "SciPy Journal" was going to be a journal about SciPy, rather than a journal about science with Python. I should have said so, but couldn't figure out a way to do it so that I didn't come across as one of those paranoid people that didn't want to "give-up" their code... which I am...
On Mon, 02 Oct 2006, Travis Oliphant apparently wrote:
My preferred approach is to have this divided into two Journals (call them A and B for now) which clarify the difference in novelty.
The "A" journal would discuss contributions where a novel algorithm, implementation, or idea is expressed while the "B" contributions are module-style Python implementations of previously-known algorithms. Another candidate for the "A" journal might be documentation and "packaging" of several "B" contributions into something that could be a SciPy sub-package.
Here is an alternate proposal: have one journal with well designated sections. If desirable, a section that experiences large growth in quality submissions can be split off later. Until then, all citations will be to a single journal, which will be helpful for name recognition (which is needed for several reasons). Four of the sections could simply be called "Algorithms", "Packages", "Implementations", and "Reviews". Each can have specific criteria for submission. Oddly enough, I do not think the name "Computational Science" is taken, and it might be nice ... Finally, to reduce the editor's burden, it will be important to limit the formats allowed for submission. If it is to be a free journal, authors must typeset articles themselves. I recommend requiring LaTeX and allowing the use of the listings package. This may be overkill, although some math journal(s) have almost fully automated the publication process this way. Also allowing restructured text, which has a LaTeX writer, may be useful---I do not know. Cheers, Alan Isaac
Alan G Isaac wrote:
Oddly enough, I do not think the name "Computational Science" is taken, and it might be nice ...
google says: International Journal of Computational Science and Engineering (IJCSE) (https://www.inderscience.com/browse/index.php?journalcode=ijcse) which is too close...
Alan G Isaac wrote:
Oddly enough, I do not think the name "Computational Science" is taken, and it might be nice ...
On Mon, 02 Oct 2006, Robert Cimrman apparently wrote:
google says: International Journal of Computational Science and Engineering (IJCSE) (https://www.inderscience.com/browse/index.php?journalcode=ijcse) which is too close...
Maybe this is field specific, but I do not consider that too close. In Economics we have, just as a very small example: Review of Political Economy Review of International Political Economy Journal of Political Economy and then there are things like Political Economy Journal etc etc etc Someone will grab the name Computational Science soon. Might as well be someone virtuous ... Cheers, Alan Isaac
Alan G Isaac wrote:
Alan G Isaac wrote:
Oddly enough, I do not think the name "Computational Science" is taken, and it might be nice ...
On Mon, 02 Oct 2006, Robert Cimrman apparently wrote:
google says: International Journal of Computational Science and Engineering (IJCSE) (https://www.inderscience.com/browse/index.php?journalcode=ijcse) which is too close...
Maybe this is field specific, but I do not consider that too close. In Economics we have, just as a very small example: Review of Political Economy Review of International Political Economy Journal of Political Economy and then there are things like Political Economy Journal etc etc etc
Someone will grab the name Computational Science soon. Might as well be someone virtuous ...
Sure - I also like the proposed name :). But still, in your examples, there is always a word different, an adjective etc, while in our case it is just a subset of the other name which could cause confusion (or am I just getting paranoid?). Maybe an adjective would help (J. of Computational Software Science?? - as a non-native english speaker, I am not sure if it sounds weird or not.) Nevertheless, having a journal is a great idea! (Had Travis any bad idea ever?) cheers, r.
Hi, I would love to see something like this and am willing to help. I think it is better to have one journal with different sections. This gives more flexibility moving forward: it is easier to change sections in a journal than it is to change an entire journal. Also, I think the title should reflect an emphasis in "scientific computing" not "computational science" or more generally "computing." And I am fine having it be Python focused in both its title and official goals. I have no desire to spend time promoting scientific computing in say Perl. I think it would be a very good idea to find sponsors for this (companies, national labs, universities). I am willing to help coordinate this effort. I am also wiling to help as an editor and reviewer. Brian
Alan G Isaac wrote:
On Mon, 02 Oct 2006, Travis Oliphant apparently wrote:
My preferred approach is to have this divided into two Journals (call them A and B for now) which clarify the difference in novelty.
Here is an alternate proposal: have one journal with well designated sections.
I like it.
If desirable, a section that experiences large growth in quality submissions can be split off later. Until then, all citations will be to a single journal, which will be helpful for name recognition (which is needed for several reasons). Four of the sections could simply be called "Algorithms", "Packages", "Implementations", and "Reviews". Each can have specific criteria for submission.
I like these names to.
Finally, to reduce the editor's burden, it will be important to limit the formats allowed for submission.
Yes, that is true. This journal needs to be author edited as much as possible. -Travis
Travis Oliphant wrote:
1) Do you like the idea ?
+1
2) Could you help (i.e. as an editor, reviewer, etc.)?
+1 As concerns the name, I like the proposal of Travis (Journal of Scientific Computing with Python) but to come with something new: 'Journal of Mixed-language Computing' to stress the principal way of getting performance in Python by coding the bottleneck parts in lower level languages. r.
On 9/29/06, Travis Oliphant <oliphant@ee.byu.edu> wrote:
As most of you know, I'm at an academic institution and have a strong belief that open source should mesh better with the academic world then it currently does. One of the problems, is that it is not easy to just point to open source software as a "publication" which is one of the areas that most tenure-granting boards look at when deciding on a candidate.
To help with that a little-bit, I'd really like to get a peer-reviewed on-line journal for code contributed to SciPy started. Not to implicate him if he no longer thinks it's a good idea, but I first started thinking seriously about this at SciPy 2006 when Fernando Perez mentioned he had seen this model with a math-related software package and thought we could try something similar.
[...] I think this is an important and wortwhile idea, and I'm willing to help. However, I think it's worth clarifying the intent of this effort to make sure it really does something useful. The GAP process (I can confirm that what Robert mentioned is what I had in mind when I spoke with Travis, we can request further details from Steve Linton at some point if we deem it necessary) addresses specifically the issue of code contributions to GAP as peer-reviewed packages, without going 'all the way' into the creation of a Journal, which is an expensive proposition (at the very least from a time and effort perspective). There are basically, I think, two issues at play here: 1. How to ensure that time spent on developing open source packages which are truly useful is acknowledged by 'traditional' academia, for things like hiring and tenure reviews, promotions, grant applications (this means that any changes we target need to make their way into the funding agencies), etc. 2. The question about having a Journal, covering modern scientific computing development, practices and algorithms, and with a more or less narrow Python focus. I am convinced that #1 is critically important: I believe strongly that the open source model produces /better/ tools, more flexibly, and in a vastly more pleasant fashion, than mailing a big check every year to your favorite vendor. But if many of us want a professional research future that involves these ideas, we're going to need them to be acknowledged by the proverbial bean counters. Else, committing significant amounts of time to open source developments will look an awful lot like professional suicide. However, I am not yet convinced that #2 is the way to achieve #1. It may be a worthy goal in and of itself, and that's certainly a valid question to be discussed. But I think it's really worth sorting out whether #2 is the only, or even a good way, to accomplish #1. The editing and publication of a journal requires a vast amount of work, and for a journal to be really worth anything towards goal #1, it /must/ achieve a certain level of respectability that takes a lot of work and time. The GAP developers seem to have found that a clearly defined process of review is enough to satisfy #1 without having to create a journal. It may be worth at least considering this option before going full bore with a journal idea. If, however, a Journal is deemed necessary, then we need to make sure that the ideas behind it give it a solid chance of long-term success as a genuine contribution to the scientific computing literature. We all know there's already way too many obscure journals nobody reads, we shouldn't be contributing to that. I'm listing below a few references to various topics and ideas I've noted over time on this topic, hoping they may be of value for guiding this discussion: - One of the central points of the open source model is that it provides for true reproducibility of computational results, since users can rebuild everything down to the operating system itself if need be. The big mantra of reproducible research has for a long time been championed by Stanford's Jon Claerbout, and there is a very famous paper by Donoho and Buckheit which summarizes this in the context of Wavelab, a Matlab-based wavelet toolkit. These are the basic references: http://sepwww.stanford.edu/research/redoc/ http://www-stat.stanford.edu/~donoho/Reports/1995/wavelab.pdf I think it would be great if a new journal would emphasize these ideas as a founding principle, by making full reproducibility (which obviously requires access to source code and data) a condition for publication. I am convinced that this alone would drastically improve the S/N ratio of the journal, by eliminating all the plotting-by-photoshop papers from the get go. - There is an interesting note in a recent LWN issue: http://lwn.net/Articles/199427/ if you scroll down to the section titled "WOS4: Quality management in free content", you'll find a description of the journal Atmospheric Chemistry and Physics (http://www.copernicus.org/EGU/acp/acp.html). They use an interesting combination of traditional peer-review and a 'publish early, discuss often' approach which apparently has produced good results. Quoting from the above: When a paper is submitted, as long as it's not complete junk, it will be immediately published as a "discussion paper" on the journal's web site. It is clearly marked as an unreviewed paper, not to be taken as definitive results at that time. While the referees are reviewing the paper, others can post comments and questions as well. These others are limited to "registered scientists," since the desire is to keep the conversation at a high level. The comments become part of the permanent record stored with the paper, and they can, at times, be cited by others in their own right. The editor will consider outside comments when deciding whether the paper is to be accepted and what revisions are to be required. After using this process for five years, Atmospheric Chemistry and Physics has the highest level of citations in the field. Citations are important in the scientific world: they are an indication that a given set of research results has helped and inspired discoveries elsewhere. The high level of citations here indicates that this publication process is succeeding in attracting high-level papers and filtering out the less useful submissions. - It's always worth having a look at the PLOS process (http://www.plos.org), which has been for a few years trying to change the publication model in the biomedical community towards a more open one. I'm not sure what actual impact their journals currently have, though. - In the high energy physics community, for a long time the de facto mechanism for real communication has been the arXiv (http://arxiv.org). I think it's fair to say that in several subfields of HEP, people push for publication in 'real journals' mostly for career reasons, but by the time a paper makes its way to Physical Review or Nuclear Physics, it has long ago been digested, discussed and commented, possibly in a round of short comment papers at the arXiv itself. While the arXiv is NOT peer-reviewed and hence the need for 'real' publications for bean-counting purposes remains, the crackpots and other poor-quality work never seem to be a major problem: people tend to just silently ignore them, while the good work is quickly picked up and generates response. Over time, the arXiv has developed subfields beyond HEP, I have no idea how successful these have been in their respective communities. - I also think we should target a higher, long-term goal: improving the standards of software development in scientific work, by 'showing the way' with an emphasis on documentation, testing (unit and otherwise), clean APIs, etc. Hopefully this will be a beneficial side effect of this effort, whether done via a journal or some other process. In any case, I'm very interested in this and I'm willing to help, and I didn't mean to undermine the enthusiasm displayed so far. Regards, f
Fernando Perez wrote:
I think this is an important and wortwhile idea, and I'm willing to help. However, I think it's worth clarifying the intent of this effort to make sure it really does something useful.
Thank you. Much of what you have written I've been concerned about as well.
The GAP process (I can confirm that what Robert mentioned is what I had in mind when I spoke with Travis, we can request further details from Steve Linton at some point if we deem it necessary) addresses specifically the issue of code contributions to GAP as peer-reviewed packages, without going 'all the way' into the creation of a Journal, which is an expensive proposition (at the very least from a time and effort perspective).
There are basically, I think, two issues at play here:
1. How to ensure that time spent on developing open source packages which are truly useful is acknowledged by 'traditional' academia, for things like hiring and tenure reviews, promotions, grant applications (this means that any changes we target need to make their way into the funding agencies), etc.
2. The question about having a Journal, covering modern scientific computing development, practices and algorithms, and with a more or less narrow Python focus.
I am convinced that #1 is critically important: I believe strongly that the open source model produces /better/ tools, more flexibly, and in a vastly more pleasant fashion, than mailing a big check every year to your favorite vendor. But if many of us want a professional research future that involves these ideas, we're going to need them to be acknowledged by the proverbial bean counters. Else, committing significant amounts of time to open source developments will look an awful lot like professional suicide.
#1 is my priority as well. When I talked about a Journal I meant it as a name one could use when citing #1. In other-words the Journal is a way to archive and document the contributions to open-source packaging.
The GAP developers seem to have found that a clearly defined process of review is enough to satisfy #1 without having to create a journal. It may be worth at least considering this option before going full bore with a journal idea.
Definitely.
In any case, I'm very interested in this and I'm willing to help, and I didn't mean to undermine the enthusiasm displayed so far.
I don't think you've undermined enthusiasm. You have identified what we should be thinking about. As I've looked around at what's available, there are already a host of journals one can submit "algorithms" to. I'd like to figure out a way we can get "peer-review" credit for contributions to the open-source world by establishing some sort of peer-review process. At my university (BYU) two things are looked at for "scholarship"-points. 1) peer-review and 2) novelty. The #1 we can easily argue for with open-source software. The novelty issue is what most people looking at contributions to scientific computing with Python will probably wonder about. Is code that implements some other-wise well known algorithm in Python "novel". Here is what my University documents indicate about scholarship evidence: "It should be of high quality and contain some element of originality, either in the form of new knowledge, new understanding, fresh insight, or unique skill or interpretation" Other documents may differ, but I think a strong case can be made for a lot (but not all) of the software that gets written for SciPy has at least "unique skill or interpretation" but perhaps also "new understanding" and "fresh insight." Other factors include: the definition of "well-known" algorithm is usually quite subjective. A lot of "well-known" algorithms actually require quite a bit of decision-making / tweaking to implement in a particular language. Not only that, but the design of an interface to code is also an aspect of code writing that could be considered scholarly. I really think that having well-stated goals and a good peer-review process and someway to "cite" the work would go a long way to solving the problem of "how can I get credit for this software" -Travis
Regards,
f _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
Thanks Fernando for spending so much time thinking about this. There is another pitfall that I see lurking here that is relevant if we want to create something that the bean counters will count. I am guessing that the reason the everyone has been so enthusiastic about the idea (including myself) is that we want such a journal to exist because we want to publish articles in it. But this won't work very well if all of us are also the journal's primary organizers/editors/reviewers. From the outside, (and to the bean counters) I think that scenario will simply look like a bunch of friends in an isolated community approving of each other's work (even though it may be more than that in reality). Because of this, no matter what we do, I think it is important to build something that people outside scipy-dev@scipy.org will want to i) read and ii) publish in. The question is how to go about this. One way would be to find sponsors - organizations and individuals that could get behind the effort and provide publicity, manpower and resources. Another thing that might be important is to have a rolling publication model like that of the physics arxiv or boost - rather than a traditional model of having issues/volumes published at regular intervals. The difficulty with the traditional model is that there is pressure to have a continual flow of articles - it also creates deadlines :( Brian
Brian Granger wrote:
Thanks Fernando for spending so much time thinking about this.
There is another pitfall that I see lurking here that is relevant if we want to create something that the bean counters will count.
I am guessing that the reason the everyone has been so enthusiastic about the idea (including myself) is that we want such a journal to exist because we want to publish articles in it. But this won't work very well if all of us are also the journal's primary organizers/editors/reviewers.
I can see your point, and we definitely want to make it as broadly interesting as possible. But, naturally, it will self-select those who are interested in using Python for scientific computing. Having reviewers be publishers is not the problem. We will need more reviewers than "editors" (used in the sense of assigning reviewers to submissions).
Because of this, no matter what we do, I think it is important to build something that people outside scipy-dev@scipy.org will want to i) read and ii) publish in. The question is how to go about this.
Outside of scipy-dev definitely, outside of scipy-user? maybe, outside of both scipy-user and numpy-discussions? I don't think so. The point is to grow both scipy-user and numpy-discussions. Perhaps they become an integral part of the review process. I don't know. Maybe a forum page is better for that.
One way would be to find sponsors - organizations and individuals that could get behind the effort and provide publicity, manpower and resources.
This is a great idea.
Another thing that might be important is to have a rolling publication model like that of the physics arxiv or boost - rather than a traditional model of having issues/volumes published at regular intervals.
I'm not sure what the difference is between the arxiv "rolling" model and the traditional model modified only to the point that you publish each submission as it gets "accepted" as an issue. I haven't studied the arxiv model though. -Travis
I can see your point, and we definitely want to make it as broadly interesting as possible. But, naturally, it will self-select those who are interested in using Python for scientific computing. Having reviewers be publishers is not the problem. We will need more reviewers than "editors" (used in the sense of assigning reviewers to submissions).
Yes, we definitely need more reviewers than editors.
Because of this, no matter what we do, I think it is important to build something that people outside scipy-dev@scipy.org will want to i) read and ii) publish in. The question is how to go about this.
Outside of scipy-dev definitely, outside of scipy-user? maybe, outside of both scipy-user and numpy-discussions? I don't think so. The point is to grow both scipy-user and numpy-discussions. Perhaps they become an integral part of the review process. I don't know. Maybe a forum page is better for that.
True, if you start to include the entire Numpy community, the group does begin to cover all those who would be interested in scientific computing with python - and others as well.
Another thing that might be important is to have a rolling publication model like that of the physics arxiv or boost - rather than a traditional model of having issues/volumes published at regular intervals.
I'm not sure what the difference is between the arxiv "rolling" model and the traditional model modified only to the point that you publish each submission as it gets "accepted" as an issue. I haven't studied the arxiv model though.
The main difference I was referring to is that in a rolling model, each article is published on a web site immediately upon being accepted/edited without waiting for the next "volume" of the journal to be published. I guess the rolling model is more like a continuous stream of article, whereas the traditional model (of print journals for instance) is bunches of article packages as a monthly or bi-monthly "volume" The arxiv is different in other ways, but I didn't have those in mind.
[Brian Granger:]
Another thing that might be important is to have a rolling publication model like that of the physics arxiv or boost - rather than a traditional model of having issues/volumes published at regular intervals.
[Travis Oliphant:]
I'm not sure what the difference is between the arxiv "rolling" model and the traditional model modified only to the point that you publish each submission as it gets "accepted" as an issue. I haven't studied the arxiv model though.
[Brian Granger:]
The main difference I was referring to is that in a rolling model, each article is published on a web site immediately upon being accepted/edited without waiting for the next "volume" of the journal to be published. I guess the rolling model is more like a continuous stream of article, whereas the traditional model (of print journals for instance) is bunches of article packages as a monthly or bi-monthly "volume" The arxiv is different in other ways, but I didn't have those in mind.
You two are talking about the same thing using different words. :-) Travis's "modified traditional" model is that of JStatSoft's: as soon as an article is accepted, it is immediately published on the web as it's own "issue". A "volume" is simply a fairly arbitrary number of consecutive "issues". In the "really traditional" model, maybe a dozen or so articles are published every n months in an "issue" and a "volume" is simply a year's worth of "issues". The "modified traditional" model is just as much a rolling model as the arxiv's, it just (ab)uses the traditional "volume/issue" terminology to make the citations look more acceptable. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
I found some resources that may be of interest: A document on the planning of an open access journal (business models, tips, etc.) http://www.soros.org/openaccess/oajguides/html/business_planning.htm Software to create and manage online open access repositories for articles: Eprints <http://software.eprints.org/> (from Southampton University, perl) DSpace <http://www.dspace.org/> (from MIT, java) CDSWare <http://cdsware.cern.ch/> (from CERN, python) FEDORA <http://www.fedora.info/> (from Cornell and U. of Virginia, java) Copernicus, a company specializing in the publication of open access journals. http://www.copernicus.org/COPERNICUS/publications/copernicus_strategies.html Cheers, David 2006/10/3, Brian Granger <ellisonbg.net@gmail.com>:
Thanks Fernando for spending so much time thinking about this.
There is another pitfall that I see lurking here that is relevant if we want to create something that the bean counters will count.
I am guessing that the reason the everyone has been so enthusiastic about the idea (including myself) is that we want such a journal to exist because we want to publish articles in it. But this won't work very well if all of us are also the journal's primary organizers/editors/reviewers. From the outside, (and to the bean counters) I think that scenario will simply look like a bunch of friends in an isolated community approving of each other's work (even though it may be more than that in reality).
Because of this, no matter what we do, I think it is important to build something that people outside scipy-dev@scipy.org will want to i) read and ii) publish in. The question is how to go about this. One way would be to find sponsors - organizations and individuals that could get behind the effort and provide publicity, manpower and resources.
Another thing that might be important is to have a rolling publication model like that of the physics arxiv or boost - rather than a traditional model of having issues/volumes published at regular intervals. The difficulty with the traditional model is that there is pressure to have a continual flow of articles - it also creates deadlines :(
Brian _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
Hi, This is an online statistical journal that appears to address some of the aspects being discussed. Regards Bruce ABOUT INTERNATIONAL JOURNAL OF BIOSTATISTICS 'The International Journal of Biostatistics' covers the entire range of biostatistics, from theoretical advances to relevant and sensible translations of a practical problem into a statistical framework, including advances in biostatistical computing. Electronic publication also allows for data and software code to be appended, and opens the door for reproducible research allowing readers to easily replicate analyses described in a paper. Both original research and review articles will be warmly received, as will articles applying sound statistical methods to practical problems. For more details, or to submit your next paper, visit: http://www.bepress.com/ijb
participants (16)
-
Alan G Isaac -
Bill Baxter -
Brian Granger -
Bruce Southey -
David Cournapeau -
David Huard -
Eric Jonas -
Fernando Perez -
Jonathan Guyer -
Prabhu Ramachandran -
Robert Cimrman -
Robert Kern -
Stefan van der Walt -
Steven H. Rogers -
Travis Oliphant -
Travis Oliphant