Accessible SciPy Request for Comments #1 Joseph Harrington Perry Greenfield At SciPy '04 we had an animated Birds-of-a-Feather session about making SciPy more accessible to the general scientific user. The session and the many conversations that followed cleared up a lot of issues and myths about SciPy. Among a core group of about a dozen participants, a consensus began to emerge about what needs to be done. The purpose of this document is to lay out our understanding of that consensus for your comment (whether or not you have been involved up to now), to announce our intention to make this vision a reality, and to solicit everyone's participation and support. Many hands make light work, and many brains, properly channeled, make great work. We thus introduce the Accessible SciPy (ASP) project. Please take a moment to look at the BoF presentation, minutes, and summary at http://www.scipy.org/wikis/scipy04/ConferenceSchedule. With those documents as background, we lay out the problem, our dream, and a path to get there. THE PROBLEM Python is wonderful for doing scientific computation. Those who try it are thrilled by the freedom from arbitrary limitations, legacy syntaxes, and license managers. Some of the applications and demos are stunning. At the same time, the experience of a new user who comes to SciPy "cold" is awful. There is little in the way of instructions for installing it and running it, even to the "Hello, world!" stage, and much of that information is outdated. There is a lengthy description of the build process, which most users could never follow even if they wanted to. It is not even clear what components are in SciPy, what else is needed, and what the add-ons do. It is to the millions of such potential users (which includes every high-school science student in the world) that the current effort is directed. We need to say at this point that we are not picking on Enthought. In fact, what's on their servers works a great deal better than they advertise. One of us was able to do a "yum install SciPy" on a laptop at the conference. But, the word "yum" appears nowhere on the web site, nor does the URL for the repository. We need to take the resources being made available by Enthought and create from them the user experience SciPy needs to thrive. WHAT'S EXPECTED BY NEW USERS New users who are not "bit-heads" expect a first-time experience that goes something like this: 1. They hear about SciPy and are somehow induced to find the site and visit. 2. They see a brief front page that says what SciPy is, lists some features, and points to a page with more info. There is a reasonable number of relevant links, all clearly laid out on a page that is not too busy. Words on the front page invite them to download and try the software. 3. They click "download", and go to a friendly page that offers a binary install package for the current version of the software for their machine (and others, but they don't care about that). It also offers instructions for installing the package on their platform. For platforms like Red Hat and Debian Linux, the instructions say what lines to add to yum.conf or its Debian equivalent, though the option to download the binary package directly also exists. The instructions end with: how to configure (if necessary, hopefully not), how to test the install, where to go for help if the install failed, a pointer to a "Getting Started With SciPy" document. They follow the instructions and it installs fine. 4. They read the "Getting Started" document, which is about 20 pages and contains: a. A brief (1/2 page) description of SciPy's key characteristics from a new user's point of view b. A get-your-feet-wet tutorial, for example: make a sine wave and a parabola, add them, and plot, read some ASCII data and plot it, do a Fourier transform of the data and plot it, read an image and display it, make a second image, add it to the first, and display it, and extract an image section, manipulate it, and display it. c. The basic (non-programming) Python syntax, including array creation, access, and manipulation. d. The key elements of Python's interaction with the operating system (e.g., the most important environment variables, how to tell it where routines are stored, etc.). e. A statement on the current state of the software, that there is flux in graphics and the underlying array package, but that the functionality we teach in the docs will not (likely) go away. f. Sources of further information and support, both in the downloaded package and on the web. 5. Encouraged by the smoothness of the experience so far, the user dives right into some topical docs (for astronomy, biology, etc.), buys a book on Python (or even SciPy), subscribes to scipy-user and a topical list, downloads add-ons, and becomes productive almost immediately. OUR SOLUTION TO THE CONUNDRUM OF OPEN DEVELOPMENT SciPy both suffers and benefits from being Open Source Software. On the one hand, many developers work on it and it is quite powerful. On the other, there is no central organization, so packages can conflict, there is duplicated work, there are holes, etc. ASP proposes to provide the organization that is lacking. Through a community-driven process, we will solicit help in resolving the issues and in implementing the solutions. We will guide new users toward what the community feels are the best packages for each basic task, so that new users experience a unified software package with clear documentation. At the same time, we will continue to provide the broadest array of application software possible, and will index all the available packages and functionality so people can make informed choices if they decide the defaults are not what they need. Mainly, we will guide users through selective documentation. For example, once we select a more capable graphics package that supports all platforms, the general user documentation will teach that package rather than xplt or gplt. The latter would continue to exist, would receive whatever support their developers and fans chose to give them, and would be listed along with all the other graphics options. They would merely not be emphasized in the tutorial documentation. Thus, nobody will lose functionality or choice. It is possible that, in the future, another package may supplant one that the documentation promotes. In that case, after due deliberation, we may decide to rewrite the documents to reflect that change. Of course, such a change would be undertaken only in compelling circumstances. WHAT'S NEEDED TO GET THERE At the BoF, we identified three key areas that needed to be worked on in order to realize the experience above: packaging, the web site, and documentation. We also identified two situations, graphics and the numeric/Numarray split, that will confuse (and therefore discourage) new users until they are resolved. We will address these first, as they frame the packaging issues. SOFTWARE ISSUES On the graphics front, there are two included packages, gplt and xplt. One works best on Unix, the other on Windows. Neither is much to write home about, and all this is acknowledged. Considerable effort has gone into Chaco, but the package has not matured as fast as some had hoped, and it is still not ready (according to most). A new player, matplotlib, is gaining popularity. With the exception of contour plots, which are being actively worked on, it seems to implement the 2D plotting capability of Matlab, with both a similar user interface to Matlab and a more capable Python-like class library. It is interactive (you can click on a screen plot and adjust it in real time). It works on all platforms. The BoF consensus was to base the new documentation on matplotlib and to de-emphasize gplt and xplt by not mentioning them in the docs at all. They would remain in SciPy indefinitely. Supporters of Chaco may advocate for the selection of that package instead. Whether the community goes for this idea will depend in large part on the state of Chaco (including package documentation and maintainability by people other than the original authors) when this decision is made. On the numeric/Numarray front, it is a problem for some that Numarray performance is worse for small arrays than that of the older numeric. Other issues exist as well. Perry Greenfield is working on getting the two packages to co-exist such that which one to use can become a run-time choice of the user, and so that package maintainers can independently choose to use either one. It is hoped that ultimately Numarray will be superior to numeric in all respects, so that numeric can be deprecated and then later removed. Thus, the BoF consensus was to document the use of Numarray for interactive work. For now, Numarray and matplotlib would not be in the SciPy core. Numarray will enter the core when the coexistence issues are resolved and the developers accept it. matplotlib may enter the core when its release schedule slows down, but at this point matplotlib releases much more frequently than SciPy and there are substantial improvements with each release. If matplotlib were in the core, users would be stuck with an incomplete and beta package until the next release of SciPy, which could be many months away. PACKAGING New users generally will not build software from source code anymore. The packaging goal is thus to produce binary packages for Linux under both RPM and the Debian Package Manager, and for Mac, Windows, and Sun using their native package managers. Support for each of these (and for other platforms) depends on interested users participating and contributing good packages on a timely basis. We appreciate any help the software developers can provide in producing the packages, but we expect that most of the packaging work will be done by volunteers who specialize in this task. The BoF consensus was to distribute SciPy, Numarray, and matplotlib as separate binary packages, possibly tied together by a meta-package that contained only dependencies (for YUM and APT users). In addition, the conference was sufficiently impressed with the interactive experience offered by IPython that it would be included until it entered the SciPy core. That move was discussed positively at the conference. Optimized matrix routines may be included, or may be available as an option, depending on our ability both to produce them in a timely fashion and to provide a simple new-user experience. DOCUMENTATION Documentation is undoubtedly the area where the largest number of work hours remain to be done. It needs rewriting, or writing for the first time, at all levels visible to users. Because of this and the beta nature of some of the packages, it makes sense to focus on shorter documents that address specific tasks, rather than something like a paper book (though that might make sense two to three years from now). It is important for docs to be well-written: clear, concise, grammatical, current, complete, correct, and visible. Thus, it makes sense to coordinate this effort closely between the writers, relevant developers, and those planning the overall effort. The approach of writing shorter docs will enable participation by a larger group of writers, thus (we hope) completing the first draft of each document sooner. It will also make updates and rewrites easier. Of course, it will be important to provide good cross-references to other documents and the web site, and to keep these up to date. Needed documentss: What is SciPy? (listing of features and included packages) Installing and Testing SciPy on Red Hat Linux Debian Gnu-Linux MAC Windows Sun Getting Started with SciPy SciPy for IDL Users, with cheat sheet SciPy for Matlab Users, with cheat sheet SciPy User's Guide SciPy Reference Manual SciPy for Astronomy, Biology, Chemistry, etc. It was noted at the BoF that Enthought has substantial documentation in its Windows package in .chm format, and that there is now an X reader for these files. Those documents will be broken out for non-Windows users and may form the bases for one or more of the documents above, assuming cooperation from Enthought on permissions and copyrights. Translations are always a good idea, but can only be obtained in a community project through the work of competent volunteers. The quality of our documentation will be vastly improved by user testing and comment, and by the participation of more than one person in the creation of each document. However, content determines form, so we should not be shy about making first drafts available, provided that we continue to work on them. We will need to agree on a set of standards for writing documents. Certainly all docs should be available in PDF and HTML, and the source should be modifiable by anyone on any platform. The Windows .chm format may also be a viable target if good tools exist to do it under Linux and Mac as well as Windows. The source form must be viable in the long term and must not leave us in a bind if the text-processing software ceases to be supported. Structured Text and LaTeX have been suggested. MSWord and other proprietary forms are out. The major need for formulae in scientific documentation may push us to LaTeX, perhaps with a standardized format, but this is open for discussion. WEB SITE Probably the easiest step is improving the web site. Enthought has provided a Plone-based site, which means that updates by many contributors will be easy and many features are already there. Much of the content needs updating, and many new areas are needed. For example: - The site says that unit tests are the main need from the community. Our needs are broader, as evidenced by the BoF conclusions. - The documentation above needs to be placed on the site as it becomes available. - Pages and mailing lists are needed for topical discussions, each managed by a field representative. - The YUM archive and other download sites need to be linked to the main pages. - The utilitarian, Plone-out-of-the-box appearance should be replaced by one with more legible buttons and some lightweight graphics that make the page attractive. - A page with demos and screen shots should be added. - Pages that present and possibly rate add-on packages are needed. - Pages of user recipes and web links are needed. - A page of demos would be nice. - Throughout, it needs to be clear who is responsible for each page and how to reach that person. - Overall, the site needs to be reorganized to give new users the experience outlined above, without making it much less useful for experienced users to navigate. Essentially, scipy.org needs to become a one-stop-shopping web portal for scientific computing in Python. By providing current, complete, and accessible information in an attractive format, the web site will show potential users that this software is serious business, that it will not be difficult to learn, and that it is actively maintained by a large user community. PRIORITIES The three areas for improvement can proceed in parallel. Since packaging issues affect the documentation, those should be resolved as soon as possible. For packaging, we need to: 1. determine whether the idea of replacing xplt and gplt with matplotlib in the docs is acceptable to the community, 2. determine whether IPython should go into the core, 3. determine whether the "coexistence" approach for resolving the numeric/Numarray split is a good idea. Then, we need to roll and field-test the binary packages. The latter effort will require volunteers, so if you want your package or platform to be supported, please sign up. For documentation, we need to agree on a common source format, an open copyright, and some stylistic guidelines. Documents are needed in roughly the order listed above, though if someone is very inspired to start on a later one, he or she should proceed. Suggestions for additional documents are welcome, as are volunteers to write them. Goals for the web site are less clear than for packaging and documentation. Some mailing list brainstorming on content and format issues would be a good idea, starting with those above. Then, we need to identify page maintainers and get them access. Several individuals stepped forward at the BoF, and a web human interface designer has volunteered to help. Again, more volunteers are needed. TOPICAL SOFTWARE As a parallel effort, there is now critical mass in several disciplines for the creation of topic-specific software. For example, in astronomy, most data come in Flexible Image Transport System (FITS) format, and there is a package written by Paul Barrett that reads and writes FITS files in Python. We wish to encourage these contributions, to provide guidance for a seamless integration with the main packages, and to help new users find these resources. We can start by hosting topical mailing lists at scipy.org and by providing a variety of web resources, including package or topic home pages, lists of packages by topic, and lists of links to packages hosted elsewhere. ADVOCACY Once the basics are in place, we can start advocating SciPy to the broader scientific community. With luck and a little effort, this should expand the group of contributors so that the remaining work can be done more quickly. Suggestions from the BoF included a conference kit so that someone can set up a booth that advocates SciPy simply by printing some posters and handouts and setting up a computer. Others included lunchtime talks in departments, killer demos, comparison web pages, and the best advocacy of all, being a good example to peers. The BoF identified the need for a user survey. Individuals can best judge when and to whom they advocate SciPy. However, it would make sense to coordinate any major presentations with the SciPy community, to ensure good synchronization with new releases, web site work, etc. CONCLUSION SciPy is a community project, and the community should coordinate and review what goes into SciPy. In that vein, this document is a genuine request for your comments, and we will hope to maintain this format in the future so that all voices can be heard. If you are willing to work on any part of this effort, please make your intentions known by adding your name and email (spam-obscured, if you like) to the ASP Wiki at http://www.scipy.org/wikis/accessible_scipy/. Since this is a community project, please be prepared to accept edits, corrections, and updates from the community, and to ask for help or to hand off your work to someone else when you are no longer able or willing to maintain it at the level or rate that is needed. The vision above is a large one. However, none of the steps is particularly large. Many projects can proceed in parallel and we gain from the completion of each task. Discussion will continue, for now, on the scipy-dev@scipy.org mailing list, with major announcements copied to scipy-user. If this topic comes to dominate the list, it may be split off into its own list, but for now we want to keep all those working on SciPy apprised of our efforts. We invite you to join us in making SciPy the environment of choice for technical computation, starting with your comments on the plan outlined above. --jh--
On Mon, 18 Oct 2004, Joe Harrington apparently wrote:
Request for Comments #1
Are user comments desirable? If so ... i. Documentation: make it clear what is copied under what circumstances, e.g. why a[1][1]=2 can fail to do what a[1,1]=2 does. ii. axes: the current SciPy change of the default axes from the Numeric and numarray defaults is a sure source of confusion to new users, who are likely to be relying on the Numeric manual supplemental SciPy guidance! Please make SciPy adopt the Numeric and numarray axes conventions and avoid needless confusion here. iii. matrices: I can see no reason for matrix objects not to have many more attributes. Why should a new user look for linalg.det(A) instead of just being able to say A.det? Similarly for many linear algebra actions on matrices, esp. decompositions. SciPy can change the packages it relies on for these services at will, as long as users get a stable interface in the matrices themselves. fwiw, Alan Isaac
Alan G Isaac wrote:
ii. axes: the current SciPy change of the default axes from the Numeric and numarray defaults is a sure source of confusion to new users, who are likely to be relying on the Numeric manual supplemental SciPy guidance! Please make SciPy adopt the Numeric and numarray axes conventions and avoid needless confusion here.
1. Neither Numeric nor numarray have a consistent axis convention to adopt. 2. The only function defined by Numeric that gets overridden by scipy and has an axis argument different is cumsum. scipy's cumsum is consistent with the axis convention chosen for scipy. 3. Numeric's documentation is incorrect wrt axis defaults for a couple of functions in any case. I'm willing to entertain the suggestion that scipy's cumsum should be renamed so as to not override Numeric's (as scipy's prod and cumprod are named so as to not override Numeric's product and cumproduct).
iii. matrices: I can see no reason for matrix objects not to have many more attributes. Why should a new user look for linalg.det(A) instead of just being able to say A.det? Similarly for many linear algebra actions on matrices, esp. decompositions. SciPy can change the packages it relies on for these services at will, as long as users get a stable interface in the matrices themselves.
The main reason is likely that no one has stepped up to implement, document, and test these attributes. Write up a patch, submit it to the issue tracker <URL:http://www.scipy.net/roundup/scipy/index>, and assign the issue to me (rkern). -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
On Mon, 18 Oct 2004, Robert Kern apparently wrote:
1. Neither Numeric nor numarray have a consistent axis convention to adopt.
In numarray (which seeks Numeric compatibility) we find axis=-1: all of fft stuff sort, argsort, argmin, argmax, trapz, diff axis=0: all of the linear algebra stuff (except diff in mlab.py, which numarray no longer documents) along with repeat, concatenate, compress, reduce, accumulate, average, take, alltrue, sometrue, zreduce, areduce, put I consider it a convention to say: all of the linear algebra stuff uses axis=0, all of fft stuff uses axis=-1 If the axis question seems pressing for the other functions, I suggest that for sort, argsort, argmin, argmax, trapz, diff SciPy can *require* an axis argument, and document this prominently. Any other deviation is going to eventually lead somebody to make a terrible mistake. It is not tenable for users to know that SciPy relies on Numeric/numarray but adopts different default behavior. And, as I said before, this complicates documentation efforts, which could be a shared effort across Numeric, numarray, and SciPy. fwiw, Alan Isaac
Alan G Isaac wrote:
On Mon, 18 Oct 2004, Robert Kern apparently wrote:
1. Neither Numeric nor numarray have a consistent axis convention to adopt.
In numarray (which seeks Numeric compatibility) we find
axis=-1: all of fft stuff sort, argsort, argmin, argmax, trapz, diff
axis=0: all of the linear algebra stuff (except diff in mlab.py, which numarray no longer documents) along with repeat, concatenate, compress, reduce, accumulate, average, take, alltrue, sometrue, zreduce, areduce, put
I consider it a convention to say: all of the linear algebra stuff uses axis=0, all of fft stuff uses axis=-1
If the axis question seems pressing for the other functions, I suggest that for sort, argsort, argmin, argmax, trapz, diff SciPy can *require* an axis argument, and document this prominently. Any other deviation is going to eventually lead somebody to make a terrible mistake.
It is not tenable for users to know that SciPy relies on Numeric/numarray but adopts different default behavior. And, as I said before, this complicates documentation efforts, which could be a shared effort across Numeric, numarray, and SciPy.
Hey Alan, I'm sorry you don't like it. Rest assured it is a well discussed topic. There is a long thread on the numpy-discussion mailing list entitled "default axis for numarray" if you would like to see more history of points from both sides. Here is a summary of the reason that axis=-1 is used in SciPy. SciPy adopted axis=-1 for all functions for the sake of consistency across the entire user experience with array operations. As people add modules, there is no question what to use for operations that reduce dimensionality along a certain axis. Also there is no question from a user's point of view what type of function they are using. They don't have to decide whether an operation is a "reduction" type operation or a time series type operation which is the current distinction for Numeric. Some people may have that rule memorized and be able to apply it across the libraries consistently, but it takes getting use. My personal experience, experience teaching and also with clients learning python has been counter to yours and suggests that a single rule across all functions is more easily learned and intuitive. The rule has the added benefit of working over contiguous regions of memory without requiring a copying of strided data into an intermediate buffer. This is more efficient and is likely why axis=-1 was chosen for fft and friends in the first place. I don't like the idea of requiring a keyword argument. Documenting more thoroughly is a good idea as with many many things in the SciPy world. thanks, eric
fwiw, Alan Isaac
_______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.net http://www.scipy.net/mailman/listinfo/scipy-dev
On Tue, 19 Oct 2004, eric jones apparently wrote:
I'm sorry you don't like it.
That is really not my objection, although of course everyone has preferences. If Numeric and numarray were to adopt the SciPy convention, I would be perfectly satisfied. (It seems hard to go that direction.) Or, if SciPy provided its own names and used its own convention, instead of over-riding Numeric (or numarray) names, I would also be happy. Especially if SciPy names were readily identifiable as such. (Of course, one can always be careful on one's own, but relying on users to be careful is generally a bad idea when it is avoidable.) The issue is consistency. Can you find me a SciPy user who never uses Numeric or numarray directly? Unless this is most of SciPy users, conflicting defaults for the same function names will eventually lead someone to make a terrible mistake. I intend that claim to be too obvious to require justification; is it? Cheers, Alan Isaac
Hey Alan, Alan G Isaac wrote:
On Tue, 19 Oct 2004, eric jones apparently wrote:
I'm sorry you don't like it.
That is really not my objection, although of course everyone has preferences.
If Numeric and numarray were to adopt the SciPy convention, I would be perfectly satisfied. (It seems hard to go that direction.)
There were long discussions about this when numarray was built.
Or, if SciPy provided its own names and used its own convention, instead of over-riding Numeric (or numarray) names, I would also be happy. Especially if SciPy names were readily identifiable as such. (Of course, one can always be careful on one's own, but relying on users to be careful is generally a bad idea when it is avoidable.)
The issue is consistency.
Yes. We both argue for that. It is just that you are for consistency across libraries (a good thing). SciPy has chosen to rectify an inconsistency (the axis argument) among functions within for numeric computing (also a good thing). The disagreement is which consideration trumps the other.
Can you find me a SciPy user who never uses Numeric or numarray directly? Unless this is most of SciPy users, conflicting defaults for the same function names will eventually lead someone to make a terrible mistake.
I agree that the inconsistency is unfortunate. So are the other inconsitencies with Numeric. For example, SciPy allows comparisons between complex numbers by comparing the real part (like Matlab).
from scipy import * a = array((1,2,3+3j)) b = array((1,3,3+4j)) a < b array([0, 1, 0],'b')
It doesn't throw exceptions and stop computation for div-by-zero and other floating point exceptions.
a = array((-1.,0.,1.)) a/0. array([ -1.#INF0000e+000, -1.#IND0000e+000, 1.#INF0000e+000])
So there are other differences besides 'axis" arguments. They number is small, but there are strong feelings about the benefit of each of the ones that stayed. The SciPy namespace includes everything from Numeric, and the intent is that you would use one or the other in your application (either use scipy for all imports, or use Numeric). Using them together indeed will result in different behaviors and therefore is a potential problem. In practice, this has not be a major issue at least for us. We just always use SciPy for everything. When Numarray was implemented by STSci, it used the same conventions used in SciPy and chose to maintain backward compatibility with Numeric in others. Experience with new users has led me to believe that the SciPy "axis" approach is more intuitive and so I was disappointed in the decisions to maintain backward compatiblity with Numeric in these areas. But not terribly so. It hasn't presented major problems. So, SciPy is destined to have a few different behaviors than either Numeric or Numarray (which also behave differently than each other in places). thanks, eric
I intend that claim to be too obvious to require justification; is it?
Cheers, Alan Isaac
_______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.net http://www.scipy.net/mailman/listinfo/scipy-dev
Alan G Isaac wrote:
Or, if SciPy provided its own names and used its own convention, instead of over-riding Numeric (or numarray) names, I would also be happy. Especially if SciPy names were readily identifiable as such. (Of course, one can always be careful on one's own, but relying on users to be careful is generally a bad idea when it is avoidable.)
Please stop using the plural. There is precisely one function whose axis default gets overridden in this manner, and that is cumsum(). Given that there are unittests for it, I can only assume that the override is deliberate and chosen with the Numeric incompatibility in mind. You can still disagree about that decision, but please constrain your comments to the specifics: namely, that you think cumsum()'s axis argument should default to 0. My position is that cumsum() should remain as it is, but it and all other changes to Numeric's behaviours should be documented. To that end, I've started a wiki page. http://www.scipy.org/wikis/accessible_scipy/NumericCompatibility -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
On Tue, 19 Oct 2004, Robert Kern apparently wrote:
Please stop using the plural. There is precisely one function whose axis default gets overridden in this manner, and that is cumsum(). Given that there are unittests for it, I can only assume that the override is deliberate and chosen with the Numeric incompatibility in mind.
I get your point, but I see this a bit differently. i. True, cumprod has a different name, but not one that differentiates it for a new user ii. sum has been marked for a possible axis change (in function_base.py) iii. On this list it has been emphasized that a. the entire Numeric namespace is imported, and b. SciPy probably intends to implement a consistent convention for axis specification, suggesting an unknown number of overrides as time goes by Look, I'm just a user, and a newbie at that. I only offered my view because the developers seem to be interested in what will make SciPy easy for newbies. So I thought it might be useful to hear from a user who got tripped up by the default axis change. (Fortunately, it was not serious.) I have had my say, and I should shut up now.
My position is that cumsum() should remain as it is, but it and all other changes to Numeric's behaviours should be documented. To that end, I've started a wiki page. http://www.scipy.org/wikis/accessible_scipy/NumericCompatibility
As a user, I will find this very useful. If you agree, I will add cumprod to this list, although it is not technically an override. Perhaps a "false friends" category is needed. Thank you, Alan Isaac
Some thoughts on documentation:
-----Original Message----- DOCUMENTATION
Documentation is undoubtedly the area where the largest number of work hours remain to be done. It needs rewriting, or writing for the first time, at all levels visible to users. Because of this and the beta nature of some of the packages, it makes sense to focus on shorter documents that address specific tasks, rather than something like a paper book (though that might make sense two to three years from now). It is important for docs to be well-written: clear, concise, grammatical, current, complete, correct, and visible.
I am happy to help with the "grammatical" part, and with the "clear, concise" part for any author who wants suggestions in that area. The user/developer community are the best ones to create "current, complete, correct" content for docs, so I applaud this effort. (The "visible" part should be a result of the web site efforts.) I would definitely encourage anyone to share their hard-won knowledge, even if they don't have greatest writing skills. The open source peer review process can hone the text, once the content exists. Focussing on shorter documents also defers one of the factors that can make otherwise well-written documents less usable: poor organization. A short document has less opportunity to be poorly structured than a long one. A collection of short documents can be assembled into a helpful organization as it evolves.
Needed documentss:
Some of these documents already exist, but perhaps need to be refocussed.
What is SciPy? (listing of features and included packages)
The FAQ document addresses some of these issues. It doesn't have to retain its current FAQ structure if that's not helpful. (I have also thought about using Plone features to add true FAQs.)
Installing and Testing SciPy on Red Hat Linux Debian Gnu-Linux MAC Windows Sun Getting Started with SciPy
There is a "tutorial", but it is not what I usually think of as a tutorial, which is a step-by-step walk-through of a basic usage scenario that illustrates commonly-used functions. The existing tutorial starts out that way, but becomes more of an overview of each of the sub-packages.
SciPy for IDL Users, with cheat sheet SciPy for Matlab Users, with cheat sheet SciPy User's Guide
The current "tutorial" might morph into a general User's Guide.
SciPy Reference Manual
The existing API doc is generated using epydoc. The docstrings need much improvement. This is an area where lots of people could contribute a little bit. We would need to establish some standards to follow (can of worms, to be opened later).
SciPy for Astronomy, Biology, Chemistry, etc.
It was noted at the BoF that Enthought has substantial documentation in its Windows package in .chm format, and that there is now an X reader for these files. Those documents will be broken out for non-Windows users and may form the bases for one or more of the documents above, assuming cooperation from Enthought on permissions and copyrights.
Eric has the final word on this, but I would think that any use of existing content that conforms to SciPy's BSD-style license would be acceptable. We can probably come up with a documentation-oriented variant of the license, like FreeBSD has (http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/LEGALNOTICE.html).
Translations are always a good idea, but can only be obtained in a community project through the work of competent volunteers. The quality of our documentation will be vastly improved by user testing and comment, and by the participation of more than one person in the creation of each document. However, content determines form, so we should not be shy about making first drafts available, provided that we continue to work on them.
OOoAuthors (http://www.oooauthors.org/) is an example of an open source documentation project that we might emulate.
We will need to agree on a set of standards for writing documents. Certainly all docs should be available in PDF and HTML, and the source should be modifiable by anyone on any platform. The Windows .chm format may also be a viable target if good tools exist to do it under Linux and Mac as well as Windows. The source form must be viable in the long term and must not leave us in a bind if the text-processing software ceases to be supported. Structured Text and LaTeX have been suggested. MSWord and other proprietary forms are out. The major need for formulae in scientific documentation may push us to LaTeX, perhaps with a standardized format, but this is open for discussion.
For any given tool, there is the burden of learning to use the tool, added to the cognitive load of actually writing the document. No matter how popular the tool you pick is, there will be some potential contributors who aren't familiar with it, and have to pay that extra price in order to participate. As always, there is also a trade-off in ease-of-use vs. power and flexibility. To keep the barrier to entry low, it might make sense to use a simple tool (such as ST or RST) for short documents and a powerful tool (such as LaTeX or DocBook) for longer documents. Note that short documents can be created directly on the SciPy.org site using Plone or (with the Zope ExternalEditor plug-in) in the editor of your choice. On the other hand, if one of the goals is including these docs in the download packages, that solution might not meet it. On the third hand, many open source projects have howtos and such that are available only online. Thanks, Joe, for contributing the energy and enthusiasm to move this forward. Regards, ------------------------- Janet Swisher Senior Technical Writer Enthought, Inc. 1-512-536-1057
On Mon, 18 Oct 2004, Joe Harrington wrote:
DOCUMENTATION
<snip>
It was noted at the BoF that Enthought has substantial documentation in its Windows package in .chm format, and that there is now an X reader for these files. Those documents will be broken out for non-Windows users and may form the bases for one or more of the documents above, assuming cooperation from Enthought on permissions and copyrights.
Translations are always a good idea, but can only be obtained in a community project through the work of competent volunteers. The quality of our documentation will be vastly improved by user testing and comment, and by the participation of more than one person in the creation of each document. However, content determines form, so we should not be shy about making first drafts available, provided that we continue to work on them.
We will need to agree on a set of standards for writing documents. Certainly all docs should be available in PDF and HTML, and the source should be modifiable by anyone on any platform. The Windows .chm format may also be a viable target if good tools exist to do it under Linux and Mac as well as Windows. The source form must be viable in the long term and must not leave us in a bind if the text-processing software ceases to be supported. Structured Text and LaTeX have been suggested. MSWord and other proprietary forms are out. The major need for formulae in scientific documentation may push us to LaTeX, perhaps with a standardized format, but this is open for discussion.
I agree that MSWord is out, even OpenOffice or StarOffice would be out for me. Python documentation project is moving from using LaTeX to ReST (note that ReST is different form ST). ReST has LaTeX capabilities but only for its LaTeX output. But in principle we can generate any desired format from LaTeX. Personally, I would prefer ReST (as the future standard documentation tool in Python) or LaTeX (as a very powerful documentation tool in general both in past and future). I have put ReST before LaTeX (in other cases it would be reversed) because ReST could be used in documentation strings as well. I could use Lyx but I consider it as WYSIWYG-LaTeX which does not impress me much when using Auctex Emacs mode in working with LaTeX documents. As I have understood from earlier discussions that LaTeX may scare people away from writing documentation for Scipy. But let me ask may be a bit harsh question: are these people able to produce good quality technical documentation for Scipy if they reject to learn LaTeX basics? Afterall, LaTeX is not that difficult to use, especially when we can provide documentation templates to get started easy, and there are nice LaTeX frontends available for Windows as well.
PRIORITIES
The three areas for improvement can proceed in parallel. Since packaging issues affect the documentation, those should be resolved as soon as possible.
For packaging, we need to:
1. determine whether the idea of replacing xplt and gplt with matplotlib in the docs is acceptable to the community,
Fine with me (as I am not currently using any of these tools yet).
2. determine whether IPython should go into the core,
I am using it on dayly basis, so +1 from me.
3. determine whether the "coexistence" approach for resolving the numeric/Numarray split is a good idea.
First, we need to get Numarray support to Scipy and then we can make decicions on the above.
Then, we need to roll and field-test the binary packages. The latter effort will require volunteers, so if you want your package or platform to be supported, please sign up.
For documentation, we need to agree on a common source format, an open copyright, and some stylistic guidelines. Documents are needed in roughly the order listed above, though if someone is very inspired to start on a later one, he or she should proceed. Suggestions for additional documents are welcome, as are volunteers to write them.
Goals for the web site are less clear than for packaging and documentation. Some mailing list brainstorming on content and format issues would be a good idea, starting with those above. Then, we need to identify page maintainers and get them access. Several individuals stepped forward at the BoF, and a web human interface designer has volunteered to help. Again, more volunteers are needed.
This is good that people offered help at the BoF, but not enough to get real work done. I think now we should take the next step and ask specific help in person. Regards, Pearu
Pearu Peterson wrote:
2. determine whether IPython should go into the core,
I am using it on dayly basis, so +1 from me.
How exactly is this supposed to work? An IPython package under scipy_core? If so, what is the benefit? The only one that I can see is that when you install scipy_core, you also install IPython. I don't see this being much more useful than putting the IPython download link right next to scipy's on the download page. IPython seems to be orthogonal to scipy. I'd be -1 on making IPython depend on scipy or scipy_core. IPython's potential audience is much wider than scipy's. Looking the other way, I don't think scipy would benefit much from having IPython as part of it. Perhaps when IPython gets rewritten to be a more flexible library for building interactive environments, then it would make some sense to bundle IPython into scipy. But for now, I'm -1. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
"RK" == Robert Kern <rkern@ucsd.edu> writes:
RK> IPython seems to be orthogonal to scipy. I'd be -1 on making RK> IPython depend on scipy or scipy_core. IPython's potential RK> audience is much wider than scipy's. Looking the other way, I With all due respect to Fernando, I agree. RK> don't think scipy would benefit much from having IPython as RK> part of it. Perhaps when IPython gets rewritten to be a more RK> flexible library for building interactive environments, then RK> it would make some sense to bundle IPython into scipy. Actually, that only makes the case stronger for it to be separate from scipy. I think all that needs to be done is for IPython to be bundled along with Enthon. Installing IPython on non-MS platforms is a breeze. Perhaps a yum archive or something for those who don't want to mess with source tarballs or GNU/stow[1]. Ideally, IPython should be the default Python shell, so it should really become part of Python. cheers, prabhu [1] -- I honestly think GNU/stow is a vastly underused piece of software. Even on a Debian testing system I need to build packages from source and GNU/stow makes managing these source based packages almost totally painless.
Prabhu Ramachandran wrote:
"RK" == Robert Kern <rkern@ucsd.edu> writes:
RK> IPython seems to be orthogonal to scipy. I'd be -1 on making RK> IPython depend on scipy or scipy_core. IPython's potential RK> audience is much wider than scipy's. Looking the other way, I
With all due respect to Fernando, I agree.
No need to apologize, since I also agree :) I think the issue is one of 'advertising' ipython as the default environment for new scientific users, not so much one of software architecture. And this goal can be achieved simply via packaging and documentation, without the need to make either tool a dependency for the other (scipy & ipython).
RK> don't think scipy would benefit much from having IPython as RK> part of it. Perhaps when IPython gets rewritten to be a more RK> flexible library for building interactive environments, then RK> it would make some sense to bundle IPython into scipy.
Actually, that only makes the case stronger for it to be separate from scipy. I think all that needs to be done is for IPython to be bundled along with Enthon. Installing IPython on non-MS platforms is a breeze. Perhaps a yum archive or something for those who don't want to mess with source tarballs or GNU/stow[1].
Now that ipython works really well under Windows, I do think it would be nice if Enthon included it by default (since it already, I think, provides ctypes and the win32 stuff ipython needs). This would make the out-of-box experience as click-and-run as possible for new users. For non-MS platforms, as you say ipython is already pretty easy to install. I already provide RPMs on scipy.org, and it would thus be trivial to copy/link those over to the official yum repo. There are fink, debian and BSD maintainers for ipython, and this is already prominently displayed on the ipython main page. So I think with respect to ipython, if there is (and there seems to be) consensus that people want it to be publicized as the 'best default shell' for scientific use, the necessary steps would only be: 1. Include it in the Python/Enthought edition bundle for Win32 users. 2. Link/copy my rpms on the yum scipy.org repo, so that yum users only have to: a) add scipy.org to their yum.conf file b) yum install scipy ipython 3. Mention it in the (yet to be written) newcomer's intro, so that people get used to a good interactive environment from day 1 instead of the crippled python shell. This only needs to be a brief section (a couple of paragraphs, which I can write), since ipython already is very extensively documented. We'd just mention what it does, and how to get it installed together with the rest of scipy. Of all the much more challenging issues the ASP project faces, ipython is one of the really easy ones. The three steps above require very little work, since most of it is already done. As I said, I'll gladly write the two paragraphs necessary for the intro guide about introducing ipython. Incidentally, I think that much of what I said here about ipython could apply almost verbatim to matplotlib. Matplotlib also wants to keep its own release cycle, but it would be good to make it as transparent as possible for new users to get it off the ground with the rest of scipy.
Ideally, IPython should be the default Python shell, so it should really become part of Python.
I only wish :) Best, f
On Sun, 31 Oct 2004, Robert Kern wrote:
Pearu Peterson wrote:
2. determine whether IPython should go into the core,
I am using it on dayly basis, so +1 from me.
How exactly is this supposed to work? An IPython package under scipy_core?
If so, what is the benefit? The only one that I can see is that when you install scipy_core, you also install IPython. I don't see this being much more useful than putting the IPython download link right next to scipy's on the download page.
IPython seems to be orthogonal to scipy. I'd be -1 on making IPython depend on scipy or scipy_core. IPython's potential audience is much wider than scipy's. Looking the other way, I don't think scipy would benefit much from having IPython as part of it. Perhaps when IPython gets rewritten to be a more flexible library for building interactive environments, then it would make some sense to bundle IPython into scipy.
Ironically, as a generic Linux user, I was thinking more of MS Windows users for whom installing packages separately rather than in all-in-one package is a big obstacle that sometimes overweights the inconvienince to ship the all-in-one package. But orthogonality is a good point, I wish we could use it more in current Scipy packages. I withdraw my +1. Pearu
Pearu Peterson wrote:
On Mon, 18 Oct 2004, Joe Harrington wrote:
DOCUMENTATION
<snip>
It was noted at the BoF that Enthought has substantial documentation in its Windows package in .chm format, and that there is now an X reader for these files. Those documents will be broken out for non-Windows users and may form the bases for one or more of the documents above, assuming cooperation from Enthought on permissions and copyrights.
Translations are always a good idea, but can only be obtained in a community project through the work of competent volunteers. The quality of our documentation will be vastly improved by user testing and comment, and by the participation of more than one person in the creation of each document. However, content determines form, so we should not be shy about making first drafts available, provided that we continue to work on them.
We will need to agree on a set of standards for writing documents. Certainly all docs should be available in PDF and HTML, and the source should be modifiable by anyone on any platform. The Windows .chm format may also be a viable target if good tools exist to do it under Linux and Mac as well as Windows. The source form must be viable in the long term and must not leave us in a bind if the text-processing software ceases to be supported. Structured Text and LaTeX have been suggested. MSWord and other proprietary forms are out. The major need for formulae in scientific documentation may push us to LaTeX, perhaps with a standardized format, but this is open for discussion.
I agree that MSWord is out, even OpenOffice or StarOffice would be out for me.
Python documentation project is moving from using LaTeX to ReST (note that ReST is different form ST). ReST has LaTeX capabilities but only for its LaTeX output. But in principle we can generate any desired format from LaTeX.
Personally, I would prefer ReST (as the future standard documentation tool in Python) or LaTeX (as a very powerful documentation tool in general both in past and future). I have put ReST before LaTeX (in other cases it would be reversed) because ReST could be used in documentation strings as well.
I could use Lyx but I consider it as WYSIWYG-LaTeX which does not impress me much when using Auctex Emacs mode in working with LaTeX documents.
As I have understood from earlier discussions that LaTeX may scare people away from writing documentation for Scipy. But let me ask may be a bit harsh question: are these people able to produce good quality technical documentation for Scipy if they reject to learn LaTeX basics? Afterall, LaTeX is not that difficult to use, especially when we can provide documentation templates to get started easy, and there are nice LaTeX frontends available for Windows as well.
Of the two options, ReST is much more preferable to me. I, and many others, use Word as my primary documenting tool and dislike LaTeX very much. Of the open options, I (and many others) would be most comfortable in OpenOffice. However, I also think that ReST has many benefits (better diffs in a code repository, editable in any editor, etc.) that are compelling. Imposing the LaTeX learning curve and infrastructure requirements on would-be contributors is too limiting in my opinion. So, of the imperfect solutions, ReST seems the most palatable to the most people. eric
On Oct 31, 2004, at 10:56 PM, eric jones wrote:
Of the two options, ReST is much more preferable to me. I, and many others, use Word as my primary documenting tool and dislike LaTeX very much. Of the open options, I (and many others) would be most comfortable in OpenOffice. However, I also think that ReST has many benefits (better diffs in a code repository, editable in any editor, etc.) that are compelling. Imposing the LaTeX learning curve and infrastructure requirements on would-be contributors is too limiting in my opinion. So, of the imperfect solutions, ReST seems the most palatable to the most people.
eric
But what about equations? Seems to me that documenting numerical and scientific libraries needs a good way of expressing equations and formulae. Does ReST support this? (My previous looks at it didn't show much in that way). Perry
On Sun, 31 Oct 2004, eric jones wrote:
Pearu Peterson wrote:
As I have understood from earlier discussions that LaTeX may scare people away from writing documentation for Scipy. But let me ask may be a bit harsh question: are these people able to produce good quality technical documentation for Scipy if they reject to learn LaTeX basics? Afterall, LaTeX is not that difficult to use, especially when we can provide documentation templates to get started easy, and there are nice LaTeX frontends available for Windows as well.
Of the two options, ReST is much more preferable to me. I, and many others, use Word as my primary documenting tool and dislike LaTeX very much. Of the open options, I (and many others) would be most comfortable in OpenOffice. However, I also think that ReST has many benefits (better diffs in a code repository, editable in any editor, etc.) that are compelling. Imposing the LaTeX learning curve and infrastructure requirements on would-be contributors is too limiting in my opinion. So, of the imperfect solutions, ReST seems the most palatable to the most people.
From reviewing ReST LaTeX capabilities again I must admit that only little progress has been made in the last 1.5 years in docutils with this respect.
If we would start using ReST right now then including math in documentations would require hackish solutions (raw::latex and mathhack). I am not sure that 'would-be contributors' (that's a pretty abstract term considering the urgent need for scipy documentations, imho) would accept that path either. The main advantages of using LaTeX are - documentation writers could start right now without worrying that the tool will change in future - it is available everywhere - different output formats are possible: PDF, HTML, PS,.. - native math markup support Disadvantages: - Eric dislikes it :) Btw, I have found that learning ReST is no way simpler than learning LaTeX. ReST is still in flux while LaTeX has been stable for decades. And setting up infrastructure for LaTeX is not that difficult. Google:"latex windows" gives lots of helpful step-by-step instructions. I have a feeling that if ReST inline support does not improve soon and we cannot decide which tool to use then there will be no Scipy docs available for other years to come. I am incling to reject 'would-be contributors' for the sake of being able to start writing Scipy docs immidiately. And last note about OpenOffice: to put it short then it's even elementary formula support sucks, pardon my language. Pearu
"PP" == Pearu Peterson <pearu@scipy.org> writes:
PP> On Sun, 31 Oct 2004, eric jones wrote: [...] PP> Btw, I have found that learning ReST is no way simpler than PP> learning LaTeX. ReST is still in flux while LaTeX has been PP> stable for decades. And setting up infrastructure for LaTeX is PP> not that difficult. Google:"latex windows" gives lots of PP> helpful step-by-step instructions. I've heard a lot about MiKTeX. For the non-Emacs users this looks good for LaTeX editing. http://www.texniccenter.org/front_content.php?idcat=26 The MediaWiki folks also appear to use some form of LaTeX markup for their equations and their math pages look pretty good. http://www.mediawiki.org When it comes to doing math right, I don't think LaTeX has any competitors in the picture. Besides, LaTeX scales really nicely too, so its easy to write a large book with it. cheers, prabhu
Pearu Peterson wrote:
On Sun, 31 Oct 2004, eric jones wrote:
Pearu Peterson wrote:
As I have understood from earlier discussions that LaTeX may scare people away from writing documentation for Scipy. But let me ask may be a bit harsh question: are these people able to produce good quality technical documentation for Scipy if they reject to learn LaTeX basics? Afterall, LaTeX is not that difficult to use, especially when we can provide documentation templates to get started easy, and there are nice LaTeX frontends available for Windows as well.
Of the two options, ReST is much more preferable to me. I, and many others, use Word as my primary documenting tool and dislike LaTeX very much. Of the open options, I (and many others) would be most comfortable in OpenOffice. However, I also think that ReST has many benefits (better diffs in a code repository, editable in any editor, etc.) that are compelling. Imposing the LaTeX learning curve and infrastructure requirements on would-be contributors is too limiting in my opinion. So, of the imperfect solutions, ReST seems the most palatable to the most people.
From reviewing ReST LaTeX capabilities again I must admit that only little
progress has been made in the last 1.5 years in docutils with this respect.
If we would start using ReST right now then including math in documentations would require hackish solutions (raw::latex and mathhack). I am not sure that 'would-be contributors' (that's a pretty abstract term considering the urgent need for scipy documentations, imho) would accept that path either.
The main advantages of using LaTeX are - documentation writers could start right now without worrying that the tool will change in future
Regardless of tool, people can start write now. Send in any document, and we'll get it coordinated into the final choice.
- it is available everywhere
OO and ReST both pass this -- and much better I might add.
- different output formats are possible: PDF, HTML, PS,.. - native math markup support
The math is the real advantage of LaTeX.
Disadvantages: - Eric dislikes it :)
LaTeX is dearly beloved by academics. It produces beautiful output for math markup. Several of you are very familiar with it. These are all very good points. My dislike is not without reason. LaTeX has a steep learning curve. The tool chain requires time to learn. I have watched many people learn and curse LaTeX as a grad student. I just talked to a (smart) guy on Friday writing his Masters thesis, and he was struggling with LaTeX, so things have changed markedly. I have worked with and met many more people in the course of school and work that are much more comfortable with other tools. The entire group I worked with as a post-doc at Duke (27 post-docs and grad students) used Word for all of their techinical papers full of equations. I don't say all this to say LaTeX sucks. It doesn't. But it does appeal to a fairly narrow niche of people, and, even in academia it is loosing market share as people move on to more user friendly tools (if less capable). Other data points. There was really one person that could build and maintain the LaTeX version of the Python docs consistently(Fred Drake). Did you ever try building it on Windows (even with MikTeX)? I'm sure it is possible, but I gave it up as not worth the effort a couple of years ago. Bruce Eckel has proven time and time again that Word is a reasonable platform for generating photo ready books. As I remember, Alex Martelli used Word for Python in a Nutshell. David Beazley started "Essential Python" in LaTeX and then abandoned it because the publisher couldn't deal with it. I think he ended up using some simple ad-hoc formatting scheme (sorta like ReST). The big difference, of course, is that none of these use equations as much as SciPy docs. In reality though, we need a lot more prose than we need equations, and imposing the LaTeX overhead on people just seems overkill. Looking at the Matlab docs, 70-90% of the docs I looked at on their website didn't have any equations in them -- only prose, code snippets, and screenshots of plots. I really want to maximize the potential of getting people to contribute this sort of thing and don't think LaTeX does that. I also don't think the list of people compotent to document SciPy is small, so saying that knowing/learning LaTeX is a useful filter for screening contributors is wrong to me. In fact new users are probably the best people to document the function set because they know what aspects tripped them up. Another consideration is, our infrastructure for technical writing at Enthought continues to grow. We would love to help coordinate and contribute to the effort, but LaTeX isn't really part of our toolset. Word, OpenOffice, and RoboHelp are. ReST is starting to be used more and more also. We are funding David Goodger to get DocUtils further along. Depending on how this goes, ReST may become even more of a tool for us. Using one of these tools makes it much easier for us to technical writers here involved in the process. I did just ask Janet Swisher about this, and she did say that she'd be willing to work with LaTeX if need be. So choosing it isn't a show stopper, just a slower-downer... If OpenOffice were a palitable solution for everyone, it would be my preference for the "users guide" with ReST used for the reference manual. People don't like OO though, so ReST seems like a reasonable alternative that minimizes the learning curve, simplifies the authoring process, and works easily on most platforms. I think it is a better option than LaTeX. If ReST can't currently do any equations at all, then that is a real limitation that would need to be overcome. Perhaps incorporating the Python code from Matplotlib that interprets LaTeX math strings into DocUtils should be a goal to solve this?? In the meantime, we could just have the latex code there without it being interpreted.
Btw, I have found that learning ReST is no way simpler than learning LaTeX.
ReST is still in flux while LaTeX has been stable for decades. And setting up infrastructure for LaTeX is not that difficult. Google:"latex windows" gives lots of helpful step-by-step instructions.
I have a feeling that if ReST inline support does not improve soon and we cannot decide which tool to use then there will be no Scipy docs available for other years to come. I am incling to reject 'would-be contributors' for the sake of being able to start writing Scipy docs immidiately.
And last note about OpenOffice: to put it short then it's even elementary formula support sucks, pardon my language.
The same can be said for many parts of LaTeX on many accounts (usability as the primary one). OpenOffice does a pretty good job on the usability side and "well enough" for equations. If you are going to write a mathematics thesis, this would be wrong. But, if we only need equations "occasionally" as suggested by Matlab's docs, then it would do the job fine. I can't imagine people haven't written docs because of a format issue. Travis' documents are all in LaTeX and checked in. Weave has its original docs in Word. People can write the prose in anything at the moment, and it will get incorporated into the final product. Perhaps the decision should be made on the following. The first person to generate 100 pages of new documentation in any format gets to pick the format. :-) Also, I'd be happy to continue a policy of "send in docs in whatever you want, we'll coordinate the final product" to maximize the chance of contributions. For the final format, I still vote for (1st) ReST or (2nd) OO. eric
On Nov 1, 2004, at 3:20 PM, eric jones wrote:
Pearu Peterson wrote:
[...]
- different output formats are possible: PDF, HTML, PS,.. - native math markup support
The math is the real advantage of LaTeX.
This really is the hard part of the decision of which to use.
Disadvantages: - Eric dislikes it :)
LaTeX is dearly beloved by academics. It produces beautiful output for math markup. Several of you are very familiar with it. These are all very good points.
My dislike is not without reason. LaTeX has a steep learning curve. The tool chain requires time to learn. I have watched many people learn and curse LaTeX as a grad student. I just talked to a (smart) guy on Friday writing his Masters thesis, and he was struggling with LaTeX, so things have changed markedly. I have worked with and met many more people in the course of school and work that are much more comfortable with other tools. The entire group I worked with as a post-doc at Duke (27 post-docs and grad students) used Word for all of their techinical papers full of equations. I don't say all this to say LaTeX sucks. It doesn't. But it does appeal to a fairly narrow niche of people, and, even in academia it is loosing market share as people move on to more user friendly tools (if less capable).
Other data points. There was really one person that could build and maintain the LaTeX version of the Python docs consistently(Fred Drake). Did you ever try building it on Windows (even with MikTeX)? I'm sure it is possible, but I gave it up as not worth the effort a couple of years ago.
I have to say that I found trying to use the Python documentation system a horror (very brittle and hard to maintain). Then again, I'm probably stupid. Still, after doing a few that way, I would definitely want to avoid using that.
Bruce Eckel has proven time and time again that Word is a reasonable platform for generating photo ready books. As I remember, Alex Martelli used Word for Python in a Nutshell. David Beazley started "Essential Python" in LaTeX and then abandoned it because the publisher couldn't deal with it. I think he ended up using some simple ad-hoc formatting scheme (sorta like ReST). The big difference, of course, is that none of these use equations as much as SciPy docs. In reality though, we need a lot more prose than we need equations, and imposing the LaTeX overhead on people just seems overkill. Looking at the Matlab docs, 70-90% of the docs I looked at on their website didn't have any equations in them -- only prose, code snippets, and screenshots of plots. I really want to maximize the potential of getting people to contribute this sort of thing and don't think LaTeX does that. I also don't think the list of people compotent to document SciPy is small, so saying that knowing/learning LaTeX is a useful filter for screening contributors is wrong to me. In fact new users are probably the best people to document the function set because they know what aspects tripped them up.
On the other hand, LaTeX's strength is equations. I don't think any of the other common tools comes close to it for either ease of use (gui equation editors are imho far more painful to use). But that is really about its only strength. Tables are a nightmare in TeX/LaTeX (and far easier with GUI's!). And it seems to me that tables are pretty painful with ReST. Sure, visually they look ok in the ASCII, but editing them is a major pain (not that I've used it for that, but it sure looks that way). So none of the tools does everything easily. Undoubtedly LaTeX is arcane for many. [...]
The same can be said for many parts of LaTeX on many accounts (usability as the primary one). OpenOffice does a pretty good job on the usability side and "well enough" for equations. If you are going to write a mathematics thesis, this would be wrong. But, if we only need equations "occasionally" as suggested by Matlab's docs, then it would do the job fine.
I can't imagine people haven't written docs because of a format issue. Travis' documents are all in LaTeX and checked in. Weave has its original docs in Word. People can write the prose in anything at the moment, and it will get incorporated into the final product.
Perhaps the decision should be made on the following. The first person to generate 100 pages of new documentation in any format gets to pick the format. :-) Also, I'd be happy to continue a policy of "send in docs in whatever you want, we'll coordinate the final product" to maximize the chance of contributions.
For the final format, I still vote for (1st) ReST or (2nd) OO.
In the end I would agree, provided that embedding LaTeX for math is permitted. Even though it won't be needed by most documents (thus many may never need to deal with it), I think it is essential to be able to have math in the document. So my preference would be 1) ReST (with embedded LaTeX for math) 2) OO 3) Pure LaTeX 4) MS Word ... Perry
On Mon, 1 Nov 2004, Perry Greenfield apparently wrote:
Tables are a nightmare in TeX/LaTeX (and far easier with GUI's!). And it seems to me that tables are pretty painful with ReST. Sure, visually they look ok in the ASCII, but editing them is a major pain (not that I've used it for that, but it sure looks that way).
I accept the general point (Word Perfect's table editor is hard to beat if you need to easily adjust table details visually) BUT i. There are GUI's for LaTeX that include table editors, including Lyx ii. There are also table editors for LaTeX, e.g., http://www.g32.org/latable/ iii. there are other nice ways to produce maintainable tables including in reST e.g. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/267662 And finally a question: is table support a crucial documentation feature? fwiw, Alan Isaac
"EJ" == eric jones <eric@enthought.com> writes:
EJ> Pearu Peterson wrote: >> The main advantages of using LaTeX are >> - documentation writers could start right now without worrying >> that the >> tool will change in future EJ> Regardless of tool, people can start write now. Send in any EJ> document, and we'll get it coordinated into the final choice. >> - it is available everywhere EJ> OO and ReST both pass this -- and much better I might add. I agree with the general pragmatic viewpoint that docs should be accepted regardless of the format and LaTeX should not be a filter. I can't agree with this sentiment that OO and reST are better than LaTeX. The answer to what tool is better depends on what you want to do. I would not write a paper/thesis in anything but LaTeX. >> - different output formats are possible: PDF, HTML, PS,.. >> - native math markup support EJ> The math is the real advantage of LaTeX. [...] EJ> LaTeX is dearly beloved by academics. It produces beautiful EJ> output for math markup. Several of you are very familiar with EJ> it. These are all very good points. Math, citations, easy referencing and labeling, consistency, elimination of the mouse, ability to work on any platform (a dumb terminal will do!) with any text editor of your choice and the ability to focus on what you want to write rather than how it looks. EJ> My dislike is not without reason. LaTeX has a steep learning EJ> curve. The tool chain requires time to learn. I have watched Writing style files may be a pain in the neck, but getting started with LaTeX is an afternoon's effort and getting fairly productive is a days work. So if you don't need to write the style file its pretty painless. For more complex stuff, it helps to have a LaTeX guru around or a good book. [...] EJ> I also don't think the list of people compotent to document EJ> SciPy is small, so saying that knowing/learning LaTeX is a EJ> useful filter for screening contributors is wrong to me. In EJ> fact new users are probably the best people to document the EJ> function set because they know what aspects tripped them up. That I agree completely with. I say, she who writes the docs picks the format, with just one note, pick something that works cross platform and not on just one, so reST and OO are good. cheers, prabhu
On Tue, 2 Nov 2004, Prabhu Ramachandran apparently wrote:
Math, citations, easy referencing and labeling
Amen. These are the glaring weaknesses of the alternatives. How important are these for the documentation effort? If they are important, how soon might reST be addressing these weaknesses? It may be worth stating the obvious: my text in this note is a sequence of perfectly valid LaTeX paragraphs. You can take your difficulties as you want them. Another plus: generating beautiful PDF effortlessly. I would spin the downside of LaTeX a bit differently: parsing the source is unfortunately taxing (judging by the efforts of others, not my own). This makes the reST plus itex attractive (although that still leaves citations, labeling, and references to be addressed), but is this about to happen or not? Cheers, Alan Isaac
On Sun, 31 Oct 2004, Pearu Peterson apparently wrote:
ReST has LaTeX capabilities but only for its LaTeX output.
Could you please point to (or provide) a detailed sample use? I did not realize things had gotten beyond an agreement on desirability: http://docutils.sourceforge.net/docs/dev/todo.html#math-markup Thanks, Alan Isaac
On Sun, 31 Oct 2004, Alan G Isaac wrote:
On Sun, 31 Oct 2004, Pearu Peterson apparently wrote:
ReST has LaTeX capabilities but only for its LaTeX output.
Could you please point to (or provide) a detailed sample use? I did not realize things had gotten beyond an agreement on desirability: http://docutils.sourceforge.net/docs/dev/todo.html#math-markup
See http://docutils.sourceforge.net/docs/user/latex.html Pearu
On Sun, 31 Oct 2004, Pearu Peterson apparently wrote:
ReST has LaTeX capabilities but only for its LaTeX output.
On Sun, 31 Oct 2004, Alan G Isaac wrote:
http://docutils.sourceforge.net/docs/dev/todo.html#math-markup
On Mon, 1 Nov 2004, Pearu Peterson apparently wrote:
Sure, but that only documents using raw, which remains awkward for inline math (doesn't it?). I was hoping that the itex directive and interpreted text role (described in the link I gave) were somehow available, or that something equivalent was. How are you handling inline math? It seems to me a nice way to handle this is needed for documentation purposes. Thanks, Alan Isaac
On Mon, 1 Nov 2004, Alan G Isaac wrote:
On Sun, 31 Oct 2004, Pearu Peterson apparently wrote:
ReST has LaTeX capabilities but only for its LaTeX output.
On Sun, 31 Oct 2004, Alan G Isaac wrote:
http://docutils.sourceforge.net/docs/dev/todo.html#math-markup
On Mon, 1 Nov 2004, Pearu Peterson apparently wrote:
Sure, but that only documents using raw, which remains awkward for inline math (doesn't it?). I was hoping that the itex directive and interpreted text role (described in the link I gave) were somehow available, or that something equivalent was.
How are you handling inline math? It seems to me a nice way to handle this is needed for documentation purposes.
Have a look at mathhack http://docutils.sourceforge.net/sandbox/cben/rolehack/README.html (I haven't tried it though) ((BTW: it seems to be possible to convert latex to HTML with MathML formulae http://www.cse.ohio-state.edu/~gurari/TeX4ht/mn.html http://www.cse.ohio-state.edu/~gurari/docs/mml-00/mml-00.html )) Best, Arnd
On Mon, 1 Nov 2004, Arnd Baecker apparently wrote:
http://docutils.sourceforge.net/sandbox/cben/rolehack/README.html
Well the name says it, doesn't it? But it's a step along the way. I was hoping Pearu was aware of major developments in reST's support of LaTeX. In particular, I was hoping that the projected Itex support had been realized, which (drawing on Mozilla's work) would mean that the math could be included in both LaTeX and XHTML files (containing MML). This would really sell me. Note that might *still* leave many problems. E.g., i. how to distinguish equation display types (probably the default environment should be 'gather'?) ii. citation in reST is extremely limited, at least as I understand it. E.g., I believe only one LaTeX citation command is supported, when the obvious thing to attempt is support of the standard natbib citation commands. How? Perhaps allowing an optional argument like [t::this] [p::orthat]. (Comment: citation is not even covered in the Primer, suggesting this is not seen as a priority.) iii. citation again: interaction with external bibliographic databases is crucial for any project that takes citation seriously. Python is well suited to this, but reST appears not to have taken steps toward it. (?) fwiw, Alan Isaac
Hi, there is one thing which is not quite clear to me yet (sorry if I missed it): There seem to be (at least?) three different types of documentation a) Manuals (like the Numeric manual) b) Tutorials (like Travis Oliphants scipy tutorial) c) doc-strings in the code Personally I think that for a) and b) LaTeX is a good choice. However, for doc-strings in the code I think that ReST is a good candidate. (Also here I would like to see mathematical formulae, which could be written in embedded LaTeX.) Let me emphasize one point which is very important according to our experience in teaching: There should be one graphical interface to access documents of all the above types. This should include a good search capability and the possibility of adding bookmarks (and more...). It should also be easy to add further documents, like the standard python documentation, wxPython docs, Ipython manual, matplotlib tutorials/manuals, ... A project which is pretty close to that is documancer, http://documancer.sourceforge.net/ (it will even work if you don't install wxMozilla, but with a less capable html renderer). I strongly recommend to try this out - we have been using it successfully for a year now. Documancer can (among others) handle html files and can also use pydoc. For each "book" a search index is generated once, which then allows very fast searching afterwards. Also adding new documents is very simple. One could pretty easily add a `ghelp search_topic` command (to IPython) which opens documancer with the results of that search. In addition for `ghelp command` one could convert the corresponding doc-string to html (including any mathematical formulae) and display it in documancer. There would be a couple of further interesting features one might like to have (eg. cross-referencing within a), b) and c) etc. ....) Another point would be that the user can add own comments, examples etc. at any place (locally stored). If then there is a well-defined path how to transfer these additions to the doc-strings and manuals this might also enhance the possibility of user contributions. I am certain, there are many pros and cons and technical problems concerning the above ideas, but maybe parts of this could become reality? Just my 2 cents, best, Arnd
I should chime in here on SciPy documentation, as I am trying to set-up a system that would allow users to contribute documentation to Python in a more fluid manner. First of all, tools such as LyX and TeXMacs make the LaTeX-is-too-hard line out of date. LyX for example is not hard to use to write documentation. Native binaries are available on Windows. People who still flounder in raw LaTeX have just not made use of the more useful front ends. With that said, I agree with Eric that we should worry less about a standard and more about the actual documentation. I also agree that there are multiple documents to be produced and these may use multiple formats. For me the most important issues are: 1) agreeing on a common command to bring up a graphical help browser (what that browser is could change over time---and even be set by the user). I like ghelp as the command to use, and feel that bringing up a chm browser is a good start. 2) improving the docstring documentation. Here is a plan for doing number 2. 1) First, use ReST in docstrings along with latex math commands where needed. i.e. $\alpha = \int_0^b f(x) \, dx$ 2) Set up a site (e.g. www.scipy.org/livedocs) which has all the docstrings from scipy available in a hierarchial form. * On each page there is documentation for a single function or class. * The documentation is separated into three parts: a) the one-liner b) the docstring to be included in the scipy code c) extra examples, documentation that will not be included in the code, but stay on the website. * Every docstring in scipy contains a link back to the appropriate livedoc page so that users can edit it and/or find out more about the function. * Ultimately the website could convert latex code to images and create a nice looking document. Gettting this working perfectly requires a bit of effort. But a simple implementation is not that hard. Comments are welcome. Another documentation effort should start creating scipy.org pages that just give examples for doing common things: * Least-square fitting * Solving systms of equations. * Optimization * Reading and writing files Perhaps here could go the slides for some of the tutorials that have been given on SciPy. -Travis O.
Travis Oliphant wrote:
I should chime in here on SciPy documentation, as I am trying to set-up a system that would allow users to contribute documentation to Python in a more fluid manner.
First of all, tools such as LyX and TeXMacs make the LaTeX-is-too-hard line out of date. LyX for example is not hard to use to write documentation. Native binaries are available on Windows. People who still flounder in raw LaTeX have just not made use of the more useful front ends.
+1 I'd like to encourage others to have a second look at lyx as a potentially good compromise. Here are some of lyx's strengths for a project like this one: - It provides a very good GUI environment where users of traditional word processing software will probably feel quite comfortable right off the bat. It is possible to generate Latex, PostScript, PDF and HTML straight from lyx's GUI without having to know a single Latex command. Knowing latex can certainly help in fine-tuning output for complex tasks, but it is not necessary for most normal needs. - Existing latex users, when in math mode, can still type \alpha and things work as usual (lyx does NOT impose the use of a gui click panel for building equations, though it provides one if desired). I grant that hardcore latex users who run a latex compiler in their head as they write may find it a bit limiting, but hopefully not to the point of driving them away. - Since its internal format is plain text, the usual dangers and annoyances of binary document formats like .doc (CVS management, easy corruption, diff/grep/etc hostile,...) are just not there. - There is an excellent bibliography management tool, incidentally written in python (pybliographic) which integrates very well with it. - It runs on the Unices, OSX (both via fink and as a native Aqua app) and Win32. - It's free. - It has all the known benefits of latex, which I'd argue go well beyond mathematical typesetting. For writing complex documents, the phenomenal structural robustness of latex (cross-referencing, numbering, bibliographies, tables of contents, etc...) should not be underestimated. - It makes some of latex's more annoying aspects a non issue: tables and figures. Adding/formatting tables in lyx (unless you want a lot of fine-tuning) is trivial, as is including figures setting all kinds of options for them. - It also has DocBook support, but I can't vouch for it, since I've never used it. I have been using lyx exclusively for all my documents since about 1998, with much success. While it's not necessarily what everybody would want to use for their daily activities, perhaps it can provide a good compromise tool which satisfies most of our needs. Incidentally, ipython's documentation is all written using lyx, and the setup.py file automatically generates PDF and HTML for distribution from the sources. Note that the manual includes large sections which are auto-generated from python files and ipython's docstrings (the whole magic section is pulled from the docstrings). While ipython's manual has no equations, it's still a 70 page document which shows what the tool can do in this context.
With that said, I agree with Eric that we should worry less about a standard and more about the actual documentation. I also agree that there are multiple documents to be produced and these may use multiple formats.
I completely agree with this. If a tool is chosen it can be publicized as a suggestion for contributors, but with a clear statement that any format is ultimately welcome. Best, f
On Mon, 1 Nov 2004, Travis Oliphant wrote:
I should chime in here on SciPy documentation, as I am trying to set-up a system that would allow users to contribute documentation to Python in a more fluid manner.
First of all, tools such as LyX and TeXMacs make the LaTeX-is-too-hard line out of date. LyX for example is not hard to use to write documentation. Native binaries are available on Windows. People who still flounder in raw LaTeX have just not made use of the more useful front ends.
Playing a bit with LyX again, I find it more acceptable general solution than OO. - there is almost zero learning curve to get started with it - it supports importing number of different file formats, including Word documents (I just tried and it works great!), making the format of contributed documentations rather free. Win32 port of LyX requires MiXTeX installed. Eric, Janet, have you tried LyX on windows? How it looks for a Windows user? There have been several documentation tools suggested for Scipy docs and there seems to be a consensus that doc writers should have a freedom to choose a format. But let me point out that this also introduces a problem of maintaining documentations. Pearu
Pearu Peterson wrote:
On Mon, 1 Nov 2004, Travis Oliphant wrote:
I should chime in here on SciPy documentation, as I am trying to set-up a system that would allow users to contribute documentation to Python in a more fluid manner.
First of all, tools such as LyX and TeXMacs make the LaTeX-is-too-hard line out of date. LyX for example is not hard to use to write documentation. Native binaries are available on Windows. People who still flounder in raw LaTeX have just not made use of the more useful front ends.
Playing a bit with LyX again, I find it more acceptable general solution than OO. - there is almost zero learning curve to get started with it - it supports importing number of different file formats, including Word documents (I just tried and it works great!), making the format of contributed documentations rather free.
Win32 port of LyX requires MiXTeX installed.
Eric, Janet, have you tried LyX on windows? How it looks for a Windows user?
I've used LyX on Windows and it seems to work fine. The version that uses Qt (the easiest one to install) has some alignment problems for on screen display of sums and integrals (the packager says it is a problem with Qt) but other than that minor annoyance works just fine. Fernando really gave a good description for LyX and I would just mention that I have also been using it very successfully since about the same time. I write all my documents using LyX now. I would like to hear from other Windows users about their success with LyX (you do have to have MikTeX installed to get other outputs besides the onscreen display and the .lyx text file). The tutorial is written in LyX. HTML could be produced but I haven't done that yet. -Travis
On Mon, 1 Nov 2004, Travis Oliphant wrote:
I should chime in here on SciPy documentation, as I am trying to set-up a system that would allow users to contribute documentation to Python in a more fluid manner.
Excellent! [...]
For me the most important issues are:
1) agreeing on a common command to bring up a graphical help browser (what that browser is could change over time---and even be set by the user). I like ghelp as the command to use, and feel that bringing up a chm browser is a good start.
What I had in mind with the graphical help-browser, was that it can be used to display manuals, tutorials etc. and doc-strings. Is there a way to supply the doc-strings (maybe converted to html from ReST, including math) to these chm browsers? Do you know of any tools (under linux) to convert html documentation to chm? ((BTW: that's what I like about documancer: it can deal with html directly, so there is no need to have documentation in different formats: for example on linux it is quite common to have the html documenation for python installed locally, so going for chm means that this contents has to be stored twice. When saying this, I really think of the graphical help browser to be _the_ way to access any type of documentation relevant for scientific computing. This might mean for one person to include documentation on PyTables, the other would like to have OpenGL stuff and so on...))
2) improving the docstring documentation.
Here is a plan for doing number 2.
1) First, use ReST in docstrings along with latex math commands where needed. i.e. $\alpha = \int_0^b f(x) \, dx$
2) Set up a site (e.g. www.scipy.org/livedocs) which has all the docstrings from scipy available in a hierarchial form. * On each page there is documentation for a single function or class. * The documentation is separated into three parts: a) the one-liner b) the docstring to be included in the scipy code c) extra examples, documentation that will not be included in the code, but stay on the website. * Every docstring in scipy contains a link back to the appropriate livedoc page so that users can edit it and/or find out more about the function.
* Ultimately the website could convert latex code to images and create a nice looking document.
Gettting this working perfectly requires a bit of effort. But a simple implementation is not that hard.
Comments are welcome.
I think this is a very good idea/approach. I have only a problem with c): I often have to work off-line (and many of our students as well), so I think it is necessary to be able to access the additional information also locally. It might be problematic to add this to the doc-strings themselves (because they could become too large, including figures etc.), but maybe one could do the following: Create a directory .scipy in the home directory of the user into which all the additional documentation can be downloaded. When invoking the help command the usual information is displayed and then it is checked if further information is available and displayed. This would have two benefits: - the users can update the documentation on their own without fiddling around with the scipy installation - a user could for himself add comments to routines easily (which he then submits to the Web-Page you suggested)
Another documentation effort should start creating scipy.org pages that just give examples for doing common things:
* Least-square fitting * Solving systms of equations. * Optimization * Reading and writing files
Perhaps here could go the slides for some of the tutorials that have been given on SciPy.
At the same time it would be good if this information was available as a separate document (a kind of "topical guide") - on the other hand your tutorial already addresses topics of this type - so should one start from that and convert it to pages on scipy.org? ((leaving aside, that the above chapters are not written yet, if I remember correctly ;-)) To summarize: - I like the idea of documentation to which additions can be made a lot - it would be very important to me if that information is available locally as well - the idea of a graphical help browser needs further discussion IMHO - of course, content is king ;-), but it has to be accessible, so one should keep this point in mind when making any decision (on chm etc.) Best, Arnd
Arnd Baecker wrote:
On Mon, 1 Nov 2004, Travis Oliphant wrote:
I should chime in here on SciPy documentation, as I am trying to set-up a system that would allow users to contribute documentation to Python in a more fluid manner.
Excellent!
Thanks for the feedback.
[...]
For me the most important issues are:
1) agreeing on a common command to bring up a graphical help browser (what that browser is could change over time---and even be set by the user). I like ghelp as the command to use, and feel that bringing up a chm browser is a good start.
What I had in mind with the graphical help-browser, was that it can be used to display manuals, tutorials etc. and doc-strings. Is there a way to supply the doc-strings (maybe converted to html from ReST, including math) to these chm browsers?
This is the idea I have as well.
Do you know of any tools (under linux) to convert html documentation to chm? ((BTW: that's what I like about documancer: it can deal with html directly, so there is no need to have documentation in different formats: for example on linux it is quite common to have the html documenation for python installed locally, so going for chm means that this contents has to be stored twice. When saying this, I really think of the graphical help browser to be _the_ way to access any type of documentation relevant for scientific computing. This might mean for one person to include documentation on PyTables, the other would like to have OpenGL stuff and so on...))
No, I know of no chm-producing tools on Linux. For me that is a downside but not a show-stopper. I need to look at documancer more.
2) improving the docstring documentation.
Here is a plan for doing number 2.
1) First, use ReST in docstrings along with latex math commands where needed. i.e. $\alpha = \int_0^b f(x) \, dx$
2) Set up a site (e.g. www.scipy.org/livedocs) which has all the docstrings from scipy available in a hierarchial form. * On each page there is documentation for a single function or class. * The documentation is separated into three parts: a) the one-liner b) the docstring to be included in the scipy code c) extra examples, documentation that will not be included in the code, but stay on the website. * Every docstring in scipy contains a link back to the appropriate livedoc page so that users can edit it and/or find out more about the function.
* Ultimately the website could convert latex code to images and create a nice looking document.
Gettting this working perfectly requires a bit of effort. But a simple implementation is not that hard.
Comments are welcome.
I think this is a very good idea/approach. I have only a problem with c): I often have to work off-line (and many of our students as well), so I think it is necessary to be able to access the additional information also locally.
Yes, the idea is that all of this extra stuff would be useful to the graphical help browser but wouldn't be in the doc-strings. So, I think we are on the same page here.
It might be problematic to add this to the doc-strings themselves (because they could become too large, including figures etc.), but maybe one could do the following: Create a directory .scipy in the home directory of the user into which all the additional documentation can be downloaded. When invoking the help command the usual information is displayed and then it is checked if further information is available and displayed.
Good ideas. I'd like to hear more input on the use of .chm files for the graphical help file. -Travis
Travis Oliphant wrote:
I'd like to hear more input on the use of .chm files for the graphical help file.
We went over some of this a few weeks ago, and I think overall people liked the idea. There are at least two graphical linux chm browsers, gnochm and xchm: http://gnochm.sourceforge.net/ http://xchm.sourceforge.net/ The problem was the creation of chms under linux, since it seems that there is no tool for this task. In that respect, the documancer approach seems appealing, as it works with regular html. Best, f
Fernando Perez wrote:
Travis Oliphant wrote:
I'd like to hear more input on the use of .chm files for the graphical help file.
We went over some of this a few weeks ago, and I think overall people liked the idea. There are at least two graphical linux chm browsers, gnochm and xchm:
http://gnochm.sourceforge.net/ http://xchm.sourceforge.net/
The problem was the creation of chms under linux, since it seems that there is no tool for this task. In that respect, the documancer approach seems appealing, as it works with regular html.
After perusing some of the documentation choices, I would say that "write" now I'm looking favorably toward documancer for a graphical help browser and lyx or tbook for producing documentation. These look like good solutions for the problems at hand. -Travis O.
Travis Oliphant schrieb:
After perusing some of the documentation choices, I would say that "write" now I'm looking favorably toward documancer for a graphical help browser and lyx or tbook for producing documentation.
These look like good solutions for the problems at hand.
+1 (I favor lyx, but simply because I'm so used to it. I have no experience with tbook) It would be great to hear a bit from Janet and Eric on this topic, to make sure that these tools are OK with their requirements/environment. But if there is no major disagreement, then we should stop discussing this particular topic and move on. There's a lot more work to do :) Cheers, f
-----Original Message----- From: scipy-dev-bounces@scipy.net [mailto:scipy-dev-bounces@scipy.net] On Behalf Of Fernando Perez
Travis Oliphant schrieb:
After perusing some of the documentation choices, I would say that "write" now I'm looking favorably toward documancer for a graphical help browser and lyx or tbook for producing documentation.
These look like good solutions for the problems at hand.
+1 (I favor lyx, but simply because I'm so used to it. I have no +experience with tbook)
It would be great to hear a bit from Janet and Eric on this topic, to make sure that these tools are OK with their requirements/environment.
Documancer sounds promising for a documentation browser solution. Regarding authoring tools, I have LyX installed, because I wanted to look at the tutorial source. I have used LaTeX in the past (10-15 years ago), and could refamiliarize myself pretty easily. However, Enthought doesn't use it for any other purpose, so I might be the only person here who is/will be familiar with it. As a tech writer, I'll use whatever tool the client wants, as long as it suits the requirements of the project. Based on all the comments thus far, I suggest using LaTeX as the "intermediate" format for (long) SciPy documents. As an author, you can generate it from LyX or from tBook, or you can just write raw LaTeX in your favorite text editor. There are LaTeX converters for MS Word and OpenOffice.org, for those who really prefer those tools. In the spirit of Election Day (and to try out our new polling plug-in), I set up a poll for documentation formats on scipy.org. You should be able to see it on the front page, or you can go to http://www.scipy.org/documentation/poll-scipy-doc-format/PlonePopoll_view2. You can vote for up to 2 choices. There is an option for "Other", but unfortunately no write-in capability. I'll leave this open for a couple of days. This poll has no "official" status, but may help to make clear the preferences of the community. --Janet
Janet Swisher schrieb:
Regarding authoring tools, I have LyX installed, because I wanted to look at the tutorial source. I have used LaTeX in the past (10-15 years ago), and could refamiliarize myself pretty easily. However, Enthought doesn't use it for any other purpose, so I might be the only person here who is/will be familiar with it.
I just want to make sure that it's clear that for most tasks, Lyx requires essentially zero latex knowledge, as the gui provides access to a lot of functionality without requiring the explicit typing of latex commands. So hopefully using lyx would not impose any additional burden on Enthought in terms of learning latex (lyx itself is similar enough to Word/OOo that hopefully its use would not be a problem). Best, f
On Tue, 2 Nov 2004, Travis Oliphant wrote:
No, I know of no chm-producing tools on Linux. For me that is a downside but not a show-stopper.
That (plus the fact that some documentation will have to be kept twice, in different formats) gives a pretty strong -1 for me. Arnd
On Mon, 1 Nov 2004, Travis Oliphant wrote:
I should chime in here on SciPy documentation, as I am trying to set-up a system that would allow users to contribute documentation to Python in a more fluid manner.
[...] I am not sure if it is helpful for what you are planning, but recently AM Kuchling announced the Annotatable Python docs, see http://pydoc.amk.ca/frame.html (to find the mail on clp search the Python Newsgroups for "pydoc.amk.ca" on http://www.python.org/search/ or http://www.amk.ca/diary/archives/cat_python.html#003336 ) I'd guess that it would be important that such a setup allows to extract the supplied information in an automatic way, so that it can be used locally as well. Arnd
-----Original Message----- From: scipy-dev-bounces@scipy.net [mailto:scipy-dev-bounces@scipy.net] On Behalf Of Pearu Peterson
As I have understood from earlier discussions that LaTeX may scare people away from writing documentation for Scipy. But let me ask may be a bit harsh question: are these people able to produce good quality technical documentation for Scipy if they reject to learn LaTeX basics?
Harsh answer: Yes. We are talking about busy people with day jobs, for whom SciPy is only a means to an end. Their knowledge of SciPy and their ability to write good quality documentation in no way depends on their ability or willingness or time available to learn or use any other specific tool. Content is king. While standardizing on a tool or toolset is important, nobody should let that stand in the way of generating content. If anybody out there has something you want to write about SciPy, please go ahead and do it using whatever tool works for you (be it HTML, ReST, LaTeX, TROFF, WordStar, or whatever). We'll find a way to get it into a format that can be shared now, and eventually into whatever format we standardize on. Also, keep in mind that one tool does not have to fit all documents. For example, it might make sense to use one format (e.g., ReST) for short documents, such as installation instructions, how-tos, and cheat sheets, and to use another format (e.g., LaTeX) for longer, book-oriented documents. A short document that evolves into requiring equations or more complex structure might have to migrate from ReST to LaTeX, but most documents could probably live their whole lives in one format or the other. ------------------------- Janet Swisher Senior Technical Writer Enthought, Inc. 1-512-536-1057
participants (11)
-
Alan G Isaac -
Arnd Baecker -
eric jones -
Fernando Perez -
Janet Swisher -
Joe Harrington -
Pearu Peterson -
Perry Greenfield -
Prabhu Ramachandran -
Robert Kern -
Travis Oliphant