--- On Thu, 8/20/09, Christopher Hanley chanley@stsci.edu wrote:
I'd like to respectfully request that we move any discussion of what to do with the numpy.char module to the numpy list.
NP, done.
I'm a little concerned about some of the assumptions that are being made about the number of users of the module. I would also like to better understand the reasons for wanting to dump it.
I think Ralf did a pretty good job of synopsizing the reasons for deprecation, but since we're moving the thread, I'll reprint them here:
0) "it gets very little use" (an assumption you presumably dispute);
1) "is pretty much undocumented" (less true than a week ago, but still true for several of the attributes, with another handful or so falling into the category of "poorly documented");
2) "probably more buggy than most other parts of NumPy" ("probably" being a euphemism, IMO);
3) "there is not a really good use-case for it" (a conjecture, but one that has yet to be challenged by counter-example);
4) it's not the first time its presence in NumPy has been questioned ("as Stefan pointed out when asking this same question last year")
5) NumPy already has a (perhaps superior) alternative ("object arrays would do nicely if one needs this functionality");
to which I'll add:
6) it is, on its face, "counter to the spirit" of NumPy.
So far, IIRC, the only reason in favor of its continued inclusion is inertia.
Let me be clear. I'm not opposed to change. However breaking other people's code just for the sake of change seems like a poor reason
So, I don't think we're proposing this "just for the sake of change"
and a mean thing to do to our customers.
Apologies, but it is not proposed maliciously.
The only other things I would add by way of "review" from the scipy-dev thread:
a compromise proposal (made by Ralf):
"Put clearly in the docs that this module exists for backwards compatibility reasons, and is not recommended for new development"
and a clarification of deprecation process (provided by Robert):
"[asked by the present author] How has deprecation in Numpy worked in the past - by dictum, vote, or consensus?
[Robert's answer] Consensus or dictum without major objection. Voting is pointless except to inform one of those."
Thanks for your time and consideration.
David Goldsmith
Thank you for your time and help, Chris
-- Christopher Hanley Senior Systems Software Engineer Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21218 (410) 338-4338
On Aug 20, 2009, at 1:35 AM, Robert Kern wrote:
On Wed, Aug 19, 2009 at 20:03, David Goldsmithd_l_goldsmith@yahoo.com
wrote:
I'm going to take it a step further: "breakage" is
always the
deterrent to change, and yet "change we must"
(i.e., "adapt or
die"). It's certainly not without precedent
- even within Numpy, I
believe - for things (though perhaps not whole
namespaces) to be
deemed "to-be-deprecated," have a warning to this
effect
established in one x.[even #].0 release, and then
be removed by the
x.[even # + 2 or + 4].0 release. How has
deprecation in Numpy
worked in the past - by dictum, vote, or
consensus?
Consensus or dictum without major objection. Voting is
pointless
except to inform one of those.
-- Robert Kern
"I have come to believe that the whole world is an
enigma, a harmless
enigma that is made terrible by our own mad attempt to
interpret it as
though it had an underlying truth." -- Umberto Eco _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
Scipy-dev mailing list Scipy-dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
Here is what I know about the chararray usage at STScI since first looking into it this morning.
It is used in PyFITS and within the COS instrument calibration code. I have not heard back from the other projects yet given most of our developers are away at this time.
It appears that the COS code can be changed easily. I am waiting to hear back from PyFITs.
Also, I do not know how many people use this particular feature. However I would point out that many people who use numpy are not also on the mailing lists. Most of the STScI do not follow the numpy list. I serve as our point of contact to the numpy community. I'm trying to gather a list of projects that use this feature and specific use cases for you. As I do not use this module myself I cannot counter your arguments at this time. If we decide to deprecate this module would we reverse this decision if we then find out that the assumptions that went into the decision were in error?
Another concern is that we told people coming from numarray to use this module. It is my opinion that at this point in the numpy release cycle that an API change needs a very strong justification. Anecdotes about the number of users, a "change or die" philosophy, and an un- articulated notion of "the spirit of numpy" do not in my consideration meet that high bar. If you would like us to provide additional documentation and tests that would be possible. I'll do it myself if that is the only think keeping the module from remaining in numpy.
This also raises the question of what else is going to go? Will recarray be removed? What about the numarray c-api compatibility layer?
Like I said earlier, I'm not opposed to change. I am just of the opinion that this isn't a simple, cut and dry decision.
For those at SciPy 2009 feel free to come yell at me and beat me with sticks. I'm the fat guy in jeans and a blue shirt sitting towards the back middle on the left.
Cheers, Chris
On Thu, Aug 20, 2009 at 10:32 AM, Christopher Hanleychanley@stsci.edu wrote:
Another concern is that we told people coming from numarray to use this module. It is my opinion that at this point in the numpy release cycle that an API change needs a very strong justification. Anecdotes about the number of users, a "change or die" philosophy, and an un- articulated notion of "the spirit of numpy" do not in my consideration meet that high bar.
I agree those are not strong reasons without more backing. What worries me the most, in both numpy and scipy, is code that nobody knows about, without any test or documentation. When it breaks, we can't fix it. That's unsustainable in the long term, because it takes a lot of time that people could spend somewhere else more useful. Especially when you have C code which does not work on some platforms, with new version of python (python 3k port, for example).
I much prefer removing code to having code that barely works and cannot be maintained. Old code that people are ready to maintain, I have nothing against.
cheers,
David
I agree with David's comments. In that theme I have removed scipy.stsci from scipy. Users get it directly from us at STScI via STSCI_PYTHON. It doesn't have any documentation in the doc system. It isn't by default in the scipy namespace. And as a recent bug report indicates they can't import it anyway.
That should clean some code up. If someday a generic image processing library is added to scipy we can consider incorporating our modules back into scipy. Until that time I would rather remove the redundancy. It also help scipy's maintainability and frees me from having to worry about a fork in the code developing.
Cheers, Chris
Hi Chris
2009/8/20 Christopher Hanley chanley@stsci.edu:
That should clean some code up. If someday a generic image processing library is added to scipy we can consider incorporating our modules back into scipy. Until that time I would rather remove the redundancy. It also help scipy's maintainability and frees me from having to worry about a fork in the code developing.
We'll be spriting on an Image Processing Scikit this weekend. If you have any functions you'd like to include, let me know.
Regards Stéfan
On Aug 20, 2009, at 7:48 PM, Stéfan van der Walt wrote:
Hi Stefan,
We'll be spriting on an Image Processing Scikit this weekend. If you have any functions you'd like to include, let me know.
Regards Stéfan
Will the Image Processing Scikit be dedicated to working with a single image or stacks of images?
Cheers, Chris
Hi Stefan,
Never mind. I just found the Sprint website and read the description. I'm sorry I hadn't found this sooner. I would have made plans to stay and help. My apologizes.
Sorry, Chris
Hi Chris & Stefan,
I will be around for most of the weekend (as I believe will Perry). I'm not sure I'll be able to contribute a lot to coding, but if there's any stuff you want to co-ordinate between STScI and Stefan's scikit, let me know if I can help. That's probably about the most useful thing I could do.
Cheers,
James.
Hi Stefan,
Never mind. I just found the Sprint website and read the description. I'm sorry I hadn't found this sooner. I would have made plans to stay and help. My apologizes.
Sorry, Chris
Christopher Hanley wrote:
Hi Stefan,
Never mind. I just found the Sprint website and read the description. I'm sorry I hadn't found this sooner. I would have made plans to stay and help. My apologizes.
Hi list,
I just saw this too and would like to misuse this thread to suggest another enhancement related to image processing you might consider: make ndimage maskedarray aware. Unfortunately we don't have any money to spend, otherwise I'd love to support this financially too. But it's a thing I'm running into (ndimage not working with missing data, that is) regularly, and for which it often is pretty hard to work out a workaround. E.g. any of the resampling (ndimage.zoom) or kernel filtering routines choke on array's with NaN's, and don't recognize masked arrays. Alas virtually all of the image data I process (satellite imagery) contains missing/bad data...
I know it probably will be a pretty involved task, as ndimage comes from numarray and seems to be largely implemented in C. But I really wanted to raise the issue now the image processing subject turns up once again, and hope some folks with more/better programming skills than me might like the idea...
Oh and I know of course ndimage is scipy, and this list is numpy. But as the image processing subject emerged here, well...
Cheers, Vincent Schut.
Sorry, Chris
Hi Vincent
2009/8/21 Vincent Schut schut@sarvision.nl:
I know it probably will be a pretty involved task, as ndimage comes from numarray and seems to be largely implemented in C. But I really wanted to raise the issue now the image processing subject turns up once again, and hope some folks with more/better programming skills than me might like the idea...
What would you like the behaviour to be? For example, how should ndimage.zoom handle these missing values?
Regards Stéfan
Stéfan van der Walt wrote:
Hi Vincent
2009/8/21 Vincent Schut schut@sarvision.nl:
I know it probably will be a pretty involved task, as ndimage comes from numarray and seems to be largely implemented in C. But I really wanted to raise the issue now the image processing subject turns up once again, and hope some folks with more/better programming skills than me might like the idea...
What would you like the behaviour to be? For example, how should ndimage.zoom handle these missing values?
Good question :-) I see 2 possibilities, both of them can be usefull in their own situations. Note that I am really not into splines mathematically, so my suggestions and terminology might not apply at all...
1. for any output cell that depends on a missing value in the input, return a missing/masked/NaN value, but (and I think this differs from the current implementation), for any output cell which could be calculated, return a proper output. Currently any array that contains one or more NaNs will give an output array full of NaNs (except for order=0, which shows this exact behaviour already). But maybe that's inherent to splines interpolation? This would at least allow input arrays with missing values (or masked arrays) to be used; this behaviour could be extended to many of the ndimage functions, like the kernel based stuff. FFT based convolutions could be another story altogether...
2. In case of zoom&co: only use the non-missing values to calculate the splines, thus effectively inter/extrapolating missing/masked values in the process. This probably raises a lot of new questions about the implementation. It would however be highly usefull for me... I don't know if a irregular grid based splines interpolation implementation exists?
What I currently do in a case like this (zooming an array with missing values) is first fill the missing values by using ndimage.generic_filter with a kernel function that averages the non-missing values in the moving window. This works as long as there are not too many missing values next to each other, however it is very slow...
I think that, if an effort like this is to be made, a thorough discussion on the possible behaviours of ndimage functions with missing values should take place on one of the numpy/scipy related mailing lists. I'm sure I'm not the only one with ideas and/or use cases for this, and I'm certainly not someone with a lot of theoretical knowledge in this area.
Regards, Vincent.
Regards Stéfan
2009/8/20 Christopher Hanley chanley@stsci.edu:
Will the Image Processing Scikit be dedicated to working with a single image or stacks of images?
Thanks for the reminder -- I have to add ImageCollection to the set of features. Fernando started working on something similar in 2006, and I've implemented a cached reader here:
http://mentat.za.net/supreme/doc/supreme.misc.io.ImageCollection-class.html
To get back to your comment: what kind of operations would you like to see supported on multiple images?
Regards Stéfan
On Aug 20, 2009, at 2:04 PM, David Cournapeau wrote:
On Thu, Aug 20, 2009 at 10:32 AM, Christopher Hanleychanley@stsci.edu wrote:
Another concern is that we told people coming from numarray to use this module. It is my opinion that at this point in the numpy release cycle that an API change needs a very strong justification. Anecdotes about the number of users, a "change or die" philosophy, and an un- articulated notion of "the spirit of numpy" do not in my consideration meet that high bar.
I agree those are not strong reasons without more backing. What worries me the most, in both numpy and scipy, is code that nobody knows about, without any test or documentation. When it breaks, we can't fix it. That's unsustainable in the long term, because it takes a lot of time that people could spend somewhere else more useful. Especially when you have C code which does not work on some platforms, with new version of python (python 3k port, for example).
The claim that "chararray" is not understood is not true. I know about the code. It's not that difficult of a piece of code. It's utility can be questioned, but it was and may still be an important part of Numarray compatibilty.
I was asked a question that I didn't have time to answer properly (it would have taken more than 5 minutes). That lack of answer does not constitute "nobody knows about" the code. It's a great idea to add to the docstring that the code was created for compatibility with numarray. But, I'm not convinced by any of the given reasons to remove the code from NumPy.
-Travis
-- Travis Oliphant Enthought Inc. 1-512-536-1057 http://www.enthought.com oliphant@enthought.com
On Thu, Aug 20, 2009 at 5:06 PM, Travis Oliphantoliphant@enthought.com wrote:
On Aug 20, 2009, at 2:04 PM, David Cournapeau wrote:
On Thu, Aug 20, 2009 at 10:32 AM, Christopher Hanleychanley@stsci.edu wrote:
Another concern is that we told people coming from numarray to use this module. It is my opinion that at this point in the numpy release cycle that an API change needs a very strong justification. Anecdotes about the number of users, a "change or die" philosophy, and an un- articulated notion of "the spirit of numpy" do not in my consideration meet that high bar.
I agree those are not strong reasons without more backing. What worries me the most, in both numpy and scipy, is code that nobody knows about, without any test or documentation. When it breaks, we can't fix it. That's unsustainable in the long term, because it takes a lot of time that people could spend somewhere else more useful. Especially when you have C code which does not work on some platforms, with new version of python (python 3k port, for example).
The claim that "chararray" is not understood is not true. I know about the code. It's not that difficult of a piece of code. It's utility can be questioned, but it was and may still be an important part of Numarray compatibilty.
I did not want to imply that chararray is unknown, sorry for the confusion.
I just wanted to say that old code is not an argument to remove something. Whether someone is willing to maintain it is a much better argument IMO.
cheers,
David
On Thu, Aug 20, 2009 at 1:32 PM, Christopher Hanley chanley@stsci.eduwrote:
Also, I do not know how many people use this particular feature. However I would point out that many people who use numpy are not also on the mailing lists. Most of the STScI do not follow the numpy list. I serve as our point of contact to the numpy community. I'm trying to gather a list of projects that use this feature and specific use cases for you.
Great. Even one good use case might change my opinion of chararray.
As I do not use this module myself I cannot counter your arguments at this time. If we decide to deprecate this module would we reverse this decision if we then find out that the assumptions that went into the decision were in error?
That would make sense.
Another concern is that we told people coming from numarray to use
this module. It is my opinion that at this point in the numpy release cycle that an API change needs a very strong justification. Anecdotes about the number of users, a "change or die" philosophy, and an un- articulated notion of "the spirit of numpy" do not in my consideration meet that high bar.
That is not very fair. I gave you four reasons for assuming there are not many users, other arguments you leave out here are the state of the code, lack of docs, tests and (most importantly) a use case.
If you would like us to provide additional documentation and tests that would be possible. I'll do it myself if that is the only think keeping the module from remaining in numpy.
Thanks a lot, that would definitely help.
How about for now we document the module as being there for numarray compatibility and not recommended for new development? Then if you turn up a good use case we add it to the docs, and if you don't we revisit the deprecation issue?
Cheers, Ralf
Sorry to resurrect a long-dead thread, but I've been continuing Chris Hanley's investigation of chararray at Space Telescope Science Institute (and the broader astronomical community) for a while and have some findings to report back.
What I've taken from this thread is that chararray is in need of a maintainer. I am able to spend some time to the cause, but first would like to clarify what it will take to make it's continued inclusion more comfortable.
Let me start with the use case. chararrays are extensively returned from pyfits (a tool to handle the standard astronomy data format). pyfits is the basis of many applications, and it would be impossible to audit all of that code. Most authors of those tools do not track numpy-discussion closely, which is why we don't hear from them on this list, but there is a great deal of pyfits-using code.
Doing some spot-checking on this code, a common thing I see is SQL-like queries on recarrays of objects. For instance, it is very common to a have a table of objects, with a "Target" column which is a string, and do something like (where c is a chararray of the 'Target' column):
subset = array[np.where(c.startswith('NGC'))]
Strictly speaking, this is a use case for "vectorized string operations", not necessarily for the chararray class as it presently stands. One could almost as easily do:
subset = array[np.where([x.startswith('NGC') for x in c])]
...and the latter is even slightly faster, since chararray currently loops in Python anyway.
Even better, though, I have some experimental code to perform the loop in C, and I get 5x speed up on a table with ~120,000 rows. If that were to be included in numpy, that's a strong argument against recommending list comprehensions in user code. The use case suggests the continued existence of vectorized string operations in numpy -- whether that continues to be chararray, or some newer/better interface + chararray for backward compatibility, is an open question. Personally I think a less object-oriented approach and just having a namespace full of vectorized string functions might be cleaner than the current situation of needing to create a view class around an ndarray. I'm suggesting something like the following, using the same example, where {STR} is some namespace we would fill with vectorized string operations:
subset = array[np.where(np.{STR}.startswith(c, 'NGC'))]
Now on to chararray as it now stands. I view chararray as really two separable pieces of functionality:
1) Convenience to perform vectorized string operations using '.method' syntax, or in some cases infix operators (+, *) 2) Implicit "rstrip"ping of values
(Note that raw ndarray's truncate values at the first NULL character, like C strings, but chararrays will strip any and all whitespace characters from the end).
Changing (2) just seems to be asking to be the source of subtle bugs. Unfortunately, there's an inconsistency between 1) and 2) in the present implementation. For example:
In [9]: a = np.char.array(['a '])
In [10]: a Out[10]: chararray(['a'], dtype='|S3')
In [11]: a[0] == 'a' Out[11]: True
In [12]: a.endswith('a') Out[12]: array([False], dtype=bool)
This is *the* design wart of chararray, IMHO, and one that's difficult to fix without breaking compatibility. It might be a worthwhile experiment to remove (2) and see how much we really break, but it would be impossible to know for sure.
Now to address the concerns iterated in this thread. Unfortunately, I don't know where this thread began before it landed on the Numpy list, so I may be missing details which would help me address them.
- "it gets very little use" (an assumption you presumably dispute);
Certainly not true from where I stand.
- "is pretty much undocumented" (less true than a week ago, but still true for several of the attributes, with another handful or so falling into the category of "poorly documented");
I don't quite understand this one -- 99% of the methods are wrappers around standard Python string methods. I don't think we should redocument those. I agree it needs a better top level docstring about its purpose (see functionalities (1) and (2) above) and its status (for backward compatibility).
- "probably more buggy than most other parts of NumPy" ("probably" being a euphemism, IMO);
Trac has these bugs. Any others?
http://projects.scipy.org/numpy/ticket/1199 http://projects.scipy.org/numpy/ticket/1200 http://projects.scipy.org/numpy/ticket/856 http://projects.scipy.org/numpy/ticket/855 http://projects.scipy.org/numpy/ticket/1231
- "there is not a really good use-case for it" (a conjecture, but one that has yet to be challenged by counter-example);
See above.
- it's not the first time its presence in NumPy has been questioned ("as Stefan pointed out when asking this same question last year")
Hopefully we're addressing that now.
- NumPy already has a (perhaps superior) alternative ("object arrays would do nicely if one needs this functionality");
No -- that gives the problem of even slower Python-looping to do vectorized string operations.
to which I'll add:
- it is, on its face, "counter to the spirit" of NumPy.
I don't quite know what this means -- but I do find the fact that it's a view class with methods a little bit clumsy. Is that what you meant?
So here's my TODO list related to all this:
1) Fix bugs in Trac 2) Improve documentation (though probably not in a method-by-method way) 3) Improve unit test coverage 4a) Create C-based vectorized string operations 4b) Refactor chararray in terms of those 4c) Design and create an interface to those methods that will be the "right way" going forward
Anything else?
Mike
Michael:
First, thank you very much for your detailed and thorough analysis and recap of the situation - it sounds to me like chararray is now in good hands! :-)
On Tue, Sep 22, 2009 at 10:58 AM, Michael Droettboom mdroe@stsci.eduwrote:
Sorry to resurrect a long-dead thread, but I've been continuing Chris
IMO, no apology necessary!
Hanley's investigation of chararray at Space Telescope Science Institute (and the broader astronomical community) for a while and have some findings to report back.
What I've taken from this thread is that chararray is in need of a maintainer. I am able to spend some time to the cause, but first would
Yes, thank you!
like to clarify what it will take to make it's continued inclusion more comfortable.
Let me start with the use case. chararrays are extensively returned from pyfits (a tool to handle the standard astronomy data format). pyfits is the basis of many applications, and it would be impossible to audit all of that code. Most authors of those tools do not track numpy-discussion closely, which is why we don't hear from them on this list, but there is a great deal of pyfits-using code.
Doing some spot-checking on this code, a common thing I see is SQL-like queries on recarrays of objects. For instance, it is very common to a have a table of objects, with a "Target" column which is a string, and do something like (where c is a chararray of the 'Target' column):
subset = array[np.where(c.startswith('NGC'))]
Strictly speaking, this is a use case for "vectorized string operations", not necessarily for the chararray class as it presently stands. One could almost as easily do:
subset = array[np.where([x.startswith('NGC') for x in c])]
...and the latter is even slightly faster, since chararray currently loops in Python anyway.
Even better, though, I have some experimental code to perform the loop in C, and I get 5x speed up on a table with ~120,000 rows. If that were to be included in numpy, that's a strong argument against recommending list comprehensions in user code. The use case suggests the continued existence of vectorized string operations in numpy -- whether that continues to be chararray, or some newer/better interface + chararray for backward compatibility, is an open question. Personally I think a less object-oriented approach and just having a namespace full of vectorized string functions might be cleaner than the current situation of needing to create a view class around an ndarray. I'm suggesting something like the following, using the same example, where {STR} is some namespace we would fill with vectorized string operations:
subset = array[np.where(np.{STR}.startswith(c, 'NGC'))]
Now on to chararray as it now stands. I view chararray as really two separable pieces of functionality:
- Convenience to perform vectorized string operations using
'.method' syntax, or in some cases infix operators (+, *) 2) Implicit "rstrip"ping of values
(Note that raw ndarray's truncate values at the first NULL character, like C strings, but chararrays will strip any and all whitespace characters from the end).
Changing (2) just seems to be asking to be the source of subtle bugs. Unfortunately, there's an inconsistency between 1) and 2) in the present implementation. For example:
In [9]: a = np.char.array(['a '])
In [10]: a Out[10]: chararray(['a'], dtype='|S3')
In [11]: a[0] == 'a' Out[11]: True
In [12]: a.endswith('a') Out[12]: array([False], dtype=bool)
This is *the* design wart of chararray, IMHO, and one that's difficult to fix without breaking compatibility. It might be a worthwhile experiment to remove (2) and see how much we really break, but it would be impossible to know for sure.
Now to address the concerns iterated in this thread. Unfortunately, I don't know where this thread began before it landed on the Numpy list, so I may be missing details which would help me address them.
- "it gets very little use" (an assumption you presumably dispute);
Certainly not true from where I stand.
I'm convinced.
- "is pretty much undocumented" (less true than a week ago, but still
true for several of the attributes, with another handful or so falling into the category of "poorly documented");
I don't quite understand this one -- 99% of the methods are wrappers
around standard Python string methods. I don't think we should
redocument those. I agree it needs a better top level docstring about
OK, that's what I needed to hear (that I don't believe anyone stated explicitly before - I'm sure I'll be corrected if I'm wrong): in that case, finishing these off is as simple as stating that in the functions' docstrings (albeit in a way compliant w/ the numpy docstring standard, of course; see below).
<snip>
- it is, on its face, "counter to the spirit" of NumPy.
I don't quite know what this means -- but I do find the fact that it's a view class with methods a little bit clumsy. Is that what you meant?
The rest of the arguments effectively become moot, but I will clarify what I meant by 6), which was simply that as I understood - and understand - it, the central purpose of numpy is to provide a fast (i.e., implemented in C), Python API for a _numerical_ multidimensional array object; it sounds like there is a need for a fast Python API for vectorized string operations, but IMO, numpy is not the place for it (maybe a sub-package in scipy? it could still use numpy "under the hood," of course); that said, my primary concern presently is getting everything that _is_ presently in numpy documented, and now, so it shall be.
So here's my TODO list related to all this:
- Fix bugs in Trac
- Improve documentation (though probably not in a method-by-method way)
So, you're volunteering to do this? Great, thanks! (Please be sure, of course, to conform to the numpy docstring standard:
http://projects.scipy.org/numpy/wiki/CodingStyleGuidelines#docstring-standar...
with clarification of referral practice, such as it is, at:
http://docs.scipy.org/numpy/Questions+Answers/#documenting-equivalent-functi... )
- Improve unit test coverage
4a) Create C-based vectorized string operations 4b) Refactor chararray in terms of those 4c) Design and create an interface to those methods that will be the "right way" going forward
Anything else?
Looks great to me!
With much thanks again!!!
DG
Mike
-- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Sep 22, 2009 at 1:58 PM, Michael Droettboom mdroe@stsci.edu wrote:
Sorry to resurrect a long-dead thread, but I've been continuing Chris Hanley's investigation of chararray at Space Telescope Science Institute (and the broader astronomical community) for a while and have some findings to report back.
Thank you for the thorough investigation, it seems clear to me now that chararray does have a purpose and is in good hands.
Now to address the concerns iterated in this thread. Unfortunately, I don't know where this thread began before it landed on the Numpy list, so I may be missing details which would help me address them.
The discussion began on the scipy list, but I think you addressed most concerns in enough detail.
- "it gets very little use" (an assumption you presumably dispute);
Certainly not true from where I stand.
- "is pretty much undocumented" (less true than a week ago, but still
true for several of the attributes, with another handful or so falling into the category of "poorly documented");
I don't quite understand this one -- 99% of the methods are wrappers around standard Python string methods. I don't think we should redocument those. I agree it needs a better top level docstring about its purpose (see functionalities (1) and (2) above) and its status (for backward compatibility).
Well, then the docstrings should say that. It can be a 4-line standard template that refers to the stdlib docs, but it would be nice to get something other than this in ipython:
charar = np.chararray(2) charar.ljust?
<stripped> Docstring: <no docstring>
- "probably more buggy than most other parts of NumPy" ("probably" being
a euphemism, IMO);
Trac has these bugs. Any others?
http://projects.scipy.org/numpy/ticket/1199 http://projects.scipy.org/numpy/ticket/1200 http://projects.scipy.org/numpy/ticket/856 http://projects.scipy.org/numpy/ticket/855 http://projects.scipy.org/numpy/ticket/1231
This one: http://article.gmane.org/gmane.comp.python.numeric.general/23638/match=chara...
Cheers, Ralf
On Tue, Sep 22, 2009 at 4:02 PM, Ralf Gommers ralf.gommers@googlemail.comwrote:
On Tue, Sep 22, 2009 at 1:58 PM, Michael Droettboom mdroe@stsci.eduwrote:
Trac has these bugs. Any others?
http://projects.scipy.org/numpy/ticket/1199 http://projects.scipy.org/numpy/ticket/1200 http://projects.scipy.org/numpy/ticket/856 http://projects.scipy.org/numpy/ticket/855 http://projects.scipy.org/numpy/ticket/1231
This one: http://article.gmane.org/gmane.comp.python.numeric.general/23638/match=chara...
Cheers, Ralf
That last one never got "promoted" to a ticket?
DG
David Goldsmith wrote:
On Tue, Sep 22, 2009 at 4:02 PM, Ralf Gommers <ralf.gommers@googlemail.com mailto:ralf.gommers@googlemail.com> wrote:
On Tue, Sep 22, 2009 at 1:58 PM, Michael Droettboom <mdroe@stsci.edu <mailto:mdroe@stsci.edu>> wrote: Trac has these bugs. Any others? http://projects.scipy.org/numpy/ticket/1199 http://projects.scipy.org/numpy/ticket/1200 http://projects.scipy.org/numpy/ticket/856 http://projects.scipy.org/numpy/ticket/855 http://projects.scipy.org/numpy/ticket/1231 This one: http://article.gmane.org/gmane.comp.python.numeric.general/23638/match=chararray Cheers, Ralf
That last one never got "promoted" to a ticket?
It's a symptom of this bug, that I created and produced a patch for yesterday:
http://projects.scipy.org/numpy/ticket/1235
Mike
Great, thanks!
DG
On Fri, Sep 25, 2009 at 6:07 AM, Michael Droettboom mdroe@stsci.edu wrote:
David Goldsmith wrote:
On Tue, Sep 22, 2009 at 4:02 PM, Ralf Gommers <ralf.gommers@googlemail.com mailto:ralf.gommers@googlemail.com>
wrote:
On Tue, Sep 22, 2009 at 1:58 PM, Michael Droettboom <mdroe@stsci.edu <mailto:mdroe@stsci.edu>> wrote: Trac has these bugs. Any others? http://projects.scipy.org/numpy/ticket/1199 http://projects.scipy.org/numpy/ticket/1200 http://projects.scipy.org/numpy/ticket/856 http://projects.scipy.org/numpy/ticket/855 http://projects.scipy.org/numpy/ticket/1231 This one:
http://article.gmane.org/gmane.comp.python.numeric.general/23638/match=chara...
Cheers, Ralf
That last one never got "promoted" to a ticket?
It's a symptom of this bug, that I created and produced a patch for yesterday:
http://projects.scipy.org/numpy/ticket/1235
Mike
-- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
I now have a rather large patch ready which addresses the following issues with chararrays. Would it be possible to get SVN commit priviledges, or would you prefer a patch file?
1) Fix bugs in Trac
http://projects.scipy.org/numpy/ticket/1199 (chararray.expandtabs broken) http://projects.scipy.org/numpy/ticket/856 (chararray __mod__ error) http://projects.scipy.org/numpy/ticket/855 (chararray __mul__ error) http://projects.scipy.org/numpy/ticket/1231 (chararray methods ignore all arguments following the first argument that evaluates to False) http://projects.scipy.org/numpy/ticket/1235 (Coercing object arrays to string arrays has surprising behaviour) http://projects.scipy.org/numpy/ticket/1240 (Casting from Unicode to String array ignores exception) http://projects.scipy.org/numpy/ticket/1241 (Array constructed with mixture of str and unicode objects fails length detection)
I can provide small individual patches for some of these if necessary, but some are interrelated and can only be fixed by the "whole enchilada".
2) Improve documentation
Every method now has a docstring, and a new page of routines has been added to the Sphinx tree.
3) Improve unit test coverage
Full line-by-line coverage of defchararray.py, as well as lots of hairy Unicode side cases.
4a) Create C-based vectorized string operations
This is benchmarking about 5x faster than the old Python-based looping on a large database of around 20k astronomical objects
4b) Refactor chararray class in terms of those
4c) Design and create an interface to those methods that will be the "right way" going forward
All vectorized string operations are now available as regular functions in the numpy.char namespace. Usage of the chararray view class is only recommended for numarray backward compatibility.
A few side notes:
http://projects.scipy.org/numpy/ticket/1200 (chararray.rstrip inconsistency)
This bug I believe should be marked as "won't fix". The inconsistent handling of trailing whitespace inconsistency is an unfortunate "feature" of the chararray class, and I am wary that fixing it may break backward compatibility. However, the new free functions in numpy.char do not have this inconsistency, so they should be recommended for new code.
http://projects.scipy.org/numpy/ticket/1240 (Casting from Unicode to String array ignores exception)
This bug probably needs review by someone deeply familiar with the low-level internals, as it affects more than just string and unicode arrays. It doesn't break any of the unit tests, for what it's worth ;)
Cheers, Mike
David Goldsmith wrote:
Great, thanks!
DG
On Fri, Sep 25, 2009 at 6:07 AM, Michael Droettboom <mdroe@stsci.edu mailto:mdroe@stsci.edu> wrote:
David Goldsmith wrote: > On Tue, Sep 22, 2009 at 4:02 PM, Ralf Gommers > <ralf.gommers@googlemail.com <mailto:ralf.gommers@googlemail.com> <mailto:ralf.gommers@googlemail.com <mailto:ralf.gommers@googlemail.com>>> wrote: > > > On Tue, Sep 22, 2009 at 1:58 PM, Michael Droettboom > <mdroe@stsci.edu <mailto:mdroe@stsci.edu> <mailto:mdroe@stsci.edu <mailto:mdroe@stsci.edu>>> wrote: > > Trac has these bugs. Any others? > > http://projects.scipy.org/numpy/ticket/1199 > http://projects.scipy.org/numpy/ticket/1200 > http://projects.scipy.org/numpy/ticket/856 > http://projects.scipy.org/numpy/ticket/855 > http://projects.scipy.org/numpy/ticket/1231 > > > This one: > http://article.gmane.org/gmane.comp.python.numeric.general/23638/match=chararray > > Cheers, > Ralf > > > That last one never got "promoted" to a ticket? It's a symptom of this bug, that I created and produced a patch for yesterday: http://projects.scipy.org/numpy/ticket/1235 Mike -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Sep 29, 2009 at 11:55 AM, Michael Droettboom mdroe@stsci.eduwrote:
I now have a rather large patch ready which addresses the following issues with chararrays. Would it be possible to get SVN commit priviledges, or would you prefer a patch file?
If you are going to maintain this part of numpy, I think you should go for the privileges. <snip>
Chuck
Michael:
Thanks so much, this is genuinely awesome! Don't forget to email Joe Harrington for your T-shirt - you more than deserve it! ;-)
A few specific comments below
On Tue, Sep 29, 2009 at 10:55 AM, Michael Droettboom mdroe@stsci.eduwrote:
I now have a rather large patch ready which addresses the following issues with chararrays. Would it be possible to get SVN commit priviledges, or would you prefer a patch file?
I hope you're granted privileges (but I can't do that - don't even have 'em myself); I'm pretty sure everyone on scipy-dev monitors this list as well, but if you have to wait more than a few days for an affirmative reply one way or the other, you should try posting there.
- Improve documentation
Every method now has a docstring, and a new page of routines has been added to the Sphinx tree.
Awesome, thanks!
4c) Design and create an interface to those methods that will be the "right way" going forward
All vectorized string operations are now available as regular functions in the numpy.char namespace. Usage of the chararray view class is only recommended for numarray backward compatibility.
Again, thanks for this. But I wonder aloud (asking long-time developers): do we have any systematic ways to steer people in such directions? (This should probably be a new thread...)
A few side notes:
http://projects.scipy.org/numpy/ticket/1200 (chararray.rstrip inconsistency)
This bug I believe should be marked as "won't fix". The inconsistent handling of trailing whitespace inconsistency is an unfortunate "feature" of the chararray class, and I am wary that fixing it may break backward compatibility. However, the new free functions in numpy.char do not have this inconsistency, so they should be recommended for new code.
OK, sounds "acceptable."
And three more times: thank you, thank you, thank you!!!
DG
Cheers, Mike
David Goldsmith wrote:
Great, thanks!
DG
On Fri, Sep 25, 2009 at 6:07 AM, Michael Droettboom <mdroe@stsci.edu mailto:mdroe@stsci.edu> wrote:
David Goldsmith wrote: > On Tue, Sep 22, 2009 at 4:02 PM, Ralf Gommers > <ralf.gommers@googlemail.com <mailto:ralf.gommers@googlemail.com> <mailto:ralf.gommers@googlemail.com <mailto:ralf.gommers@googlemail.com>>> wrote: > > > On Tue, Sep 22, 2009 at 1:58 PM, Michael Droettboom > <mdroe@stsci.edu <mailto:mdroe@stsci.edu> <mailto:mdroe@stsci.edu <mailto:mdroe@stsci.edu>>> wrote: > > Trac has these bugs. Any others? > > http://projects.scipy.org/numpy/ticket/1199 > http://projects.scipy.org/numpy/ticket/1200 > http://projects.scipy.org/numpy/ticket/856 > http://projects.scipy.org/numpy/ticket/855 > http://projects.scipy.org/numpy/ticket/1231 > > > This one: >
http://article.gmane.org/gmane.comp.python.numeric.general/23638/match=chara...
> > Cheers, > Ralf > > > That last one never got "promoted" to a ticket? It's a symptom of this bug, that I created and produced a patch for yesterday: http://projects.scipy.org/numpy/ticket/1235 Mike -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Sep 29, 2009 at 10:55 AM, Michael Droettboom mdroe@stsci.eduwrote:
- Improve documentation
Every method now has a docstring, and a new page of routines has been added to the Sphinx tree.
Um, where did you do this, 'cause it's not showing up in the doc wiki.
DG
In the source in my working copy. Is that going to cause problems? I wasn't sure if it was possible to document methods that didn't yet exist in the code in the wiki.
Mike
David Goldsmith wrote:
On Tue, Sep 29, 2009 at 10:55 AM, Michael Droettboom <mdroe@stsci.edu mailto:mdroe@stsci.edu> wrote:
2) Improve documentation Every method now has a docstring, and a new page of routines has been added to the Sphinx tree.
Um, where did you do this, 'cause it's not showing up in the doc wiki.
DG
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Sep 30, 2009 at 9:03 AM, Michael Droettboom mdroe@stsci.edu wrote:
In the source in my working copy. Is that going to cause problems? I wasn't sure if it was possible to document methods that didn't yet exist in the code in the wiki.
That is fine. New functions will automatically show up in the wiki. It
would be helpful though if you could mark them ready for review in the wiki (if they are) after they show up. Could take up to 24 hours for svn changes to propagate.
Only if you moved functions around it would be useful if you pinged Pauli after you committed them. This is a temporary problem, right now the wiki creates a new page for a moved object, and the old content (if any) has to be copied over to the new page.
Cheers, Ralf
Mike
David Goldsmith wrote:
On Tue, Sep 29, 2009 at 10:55 AM, Michael Droettboom <mdroe@stsci.edu mailto:mdroe@stsci.edu> wrote:
2) Improve documentation Every method now has a docstring, and a new page of routines has been added to the Sphinx tree.
Um, where did you do this, 'cause it's not showing up in the doc wiki.
DG
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Ralf Gommers wrote:
On Wed, Sep 30, 2009 at 9:03 AM, Michael Droettboom <mdroe@stsci.edu mailto:mdroe@stsci.edu> wrote:
In the source in my working copy. Is that going to cause problems? I wasn't sure if it was possible to document methods that didn't yet exist in the code in the wiki.
That is fine. New functions will automatically show up in the wiki. It would be helpful though if you could mark them ready for review in the wiki (if they are) after they show up. Could take up to 24 hours for svn changes to propagate.
Thanks. Will do.
Only if you moved functions around it would be useful if you pinged Pauli after you committed them. This is a temporary problem, right now the wiki creates a new page for a moved object, and the old content (if any) has to be copied over to the new page.
All of the functions that were moved were previously without docstrings in SVN, though some had docstrings (that I just now discovered) in the wiki. This may cause some hiccups, I suppose, so I'll be sure to announce when these things get committed to SVN so I know how to help straighten these things out.
Mike
Cheers, Ralf
Mike David Goldsmith wrote: > On Tue, Sep 29, 2009 at 10:55 AM, Michael Droettboom <mdroe@stsci.edu <mailto:mdroe@stsci.edu> > <mailto:mdroe@stsci.edu <mailto:mdroe@stsci.edu>>> wrote: > > 2) Improve documentation > > Every method now has a docstring, and a new page of routines has been > added to the Sphinx tree. > > > Um, where did you do this, 'cause it's not showing up in the doc wiki. > > DG > ------------------------------------------------------------------------ > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> > http://mail.scipy.org/mailman/listinfo/numpy-discussion > -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
So, Ralf (or anyone), how, if at all, should we modify the status of the existing chararray objects/methods in the wiki? Assuming you have no problem sharing them with me, Michael, I could add those docstrings you created for the existing methods, and we can promote them to "Ready for Review"; they can be "demoted" to "Unimportant" (by someone with the appropriate privileges); or I can promote them to "Being written" and add a comment that the docstring has been written, it just resides elsewhere for the time being (however, IMO they should not be left in "Needs editing" status).
DG
On Wed, Sep 30, 2009 at 7:20 AM, Michael Droettboom mdroe@stsci.edu wrote:
Ralf Gommers wrote:
On Wed, Sep 30, 2009 at 9:03 AM, Michael Droettboom <mdroe@stsci.edu mailto:mdroe@stsci.edu> wrote:
In the source in my working copy. Is that going to cause problems?
I
wasn't sure if it was possible to document methods that didn't yet exist in the code in the wiki.
That is fine. New functions will automatically show up in the wiki. It would be helpful though if you could mark them ready for review in the wiki (if they are) after they show up. Could take up to 24 hours for svn changes to propagate.
Thanks. Will do.
Only if you moved functions around it would be useful if you pinged Pauli after you committed them. This is a temporary problem, right now the wiki creates a new page for a moved object, and the old content (if any) has to be copied over to the new page.
All of the functions that were moved were previously without docstrings in SVN, though some had docstrings (that I just now discovered) in the wiki. This may cause some hiccups, I suppose, so I'll be sure to announce when these things get committed to SVN so I know how to help straighten these things out.
Mike
Cheers, Ralf
Mike David Goldsmith wrote: > On Tue, Sep 29, 2009 at 10:55 AM, Michael Droettboom <mdroe@stsci.edu <mailto:mdroe@stsci.edu> > <mailto:mdroe@stsci.edu <mailto:mdroe@stsci.edu>>> wrote: > > 2) Improve documentation > > Every method now has a docstring, and a new page of routines has been > added to the Sphinx tree. > > > Um, where did you do this, 'cause it's not showing up in the doc wiki. > > DG >
> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> > http://mail.scipy.org/mailman/listinfo/numpy-discussion > -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Sep 30, 2009 at 4:37 PM, David Goldsmith d.l.goldsmith@gmail.comwrote:
So, Ralf (or anyone), how, if at all, should we modify the status of the existing chararray objects/methods in the wiki?
Nothing has to be done until *after* Mike has committed his changes to svn. Please see my previous email for what has to happen at that point. Since Mike wrote the new docstrings it would be best if he updated the status of the wiki pages then.
Assuming you have no problem sharing them with me, Michael, I could add those docstrings you created for the existing methods,
They will show up in the wiki when they get committed to svn (presumably within a few days), so this is needless effort for the most part. If there are different changes in the wiki and svn, that will show up in the "merge" page.
The ony thing that requires manual effort is if there are changes in the wiki and the object got moved in svn.
Cheers, Ralf
DG
On Wed, Sep 30, 2009 at 7:20 AM, Michael Droettboom mdroe@stsci.eduwrote:
Ralf Gommers wrote:
On Wed, Sep 30, 2009 at 9:03 AM, Michael Droettboom <mdroe@stsci.edu mailto:mdroe@stsci.edu> wrote:
In the source in my working copy. Is that going to cause problems?
I
wasn't sure if it was possible to document methods that didn't yet exist in the code in the wiki.
That is fine. New functions will automatically show up in the wiki. It would be helpful though if you could mark them ready for review in the wiki (if they are) after they show up. Could take up to 24 hours for svn changes to propagate.
Thanks. Will do.
Only if you moved functions around it would be useful if you pinged Pauli after you committed them. This is a temporary problem, right now the wiki creates a new page for a moved object, and the old content (if any) has to be copied over to the new page.
All of the functions that were moved were previously without docstrings in SVN, though some had docstrings (that I just now discovered) in the wiki. This may cause some hiccups, I suppose, so I'll be sure to announce when these things get committed to SVN so I know how to help straighten these things out.
Mike
Cheers, Ralf
Mike David Goldsmith wrote: > On Tue, Sep 29, 2009 at 10:55 AM, Michael Droettboom <mdroe@stsci.edu <mailto:mdroe@stsci.edu> > <mailto:mdroe@stsci.edu <mailto:mdroe@stsci.edu>>> wrote: > > 2) Improve documentation > > Every method now has a docstring, and a new page of routines has been > added to the Sphinx tree. > > > Um, where did you do this, 'cause it's not showing up in the doc wiki. > > DG >
> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> > http://mail.scipy.org/mailman/listinfo/numpy-discussion > -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Sep 30, 2009 at 2:09 PM, Ralf Gommers ralf.gommers@googlemail.comwrote:
On Wed, Sep 30, 2009 at 4:37 PM, David Goldsmith d.l.goldsmith@gmail.comwrote:
So, Ralf (or anyone), how, if at all, should we modify the status of the existing chararray objects/methods in the wiki?
Nothing has to be done until *after* Mike has committed his changes to svn. Please see my previous email for what has to happen at that point. Since Mike wrote the new docstrings it would be best if he updated the status of the wiki pages then.
OK; Mike: hopefully it will be clear what you have to do to update the status (it's pretty trivial) but of course don't hesitate to email (you can do so off-list if you prefer) w/ any questions; unfortunately, AFAIK, there's no way to update the status of many docstrings all at once - you'll have to do them each individually (if you like, let me know when you've committed them and I can help - it sounds like there will be a lot); the main "silly" thing to remember is that the option to change the "Review status" only appears if you're logged in. :-)
Assuming you have no problem sharing them with me, Michael, I could add
those docstrings you created for the existing methods,
They will show up in the wiki when they get committed to svn (presumably within a few days), so this is needless effort for the most part. If there are different changes in the wiki and svn, that will show up in the "merge" page.
The ony thing that requires manual effort is if there are changes in the wiki and the object got moved in svn.
And, as above, updating the status in the Wiki. :-)
DG
Cheers, Ralf
DG
On Wed, Sep 30, 2009 at 7:20 AM, Michael Droettboom mdroe@stsci.eduwrote:
Ralf Gommers wrote:
On Wed, Sep 30, 2009 at 9:03 AM, Michael Droettboom <mdroe@stsci.edu mailto:mdroe@stsci.edu> wrote:
In the source in my working copy. Is that going to cause problems?
I
wasn't sure if it was possible to document methods that didn't yet exist in the code in the wiki.
That is fine. New functions will automatically show up in the wiki. It would be helpful though if you could mark them ready for review in the wiki (if they are) after they show up. Could take up to 24 hours for svn changes to propagate.
Thanks. Will do.
Only if you moved functions around it would be useful if you pinged Pauli after you committed them. This is a temporary problem, right now the wiki creates a new page for a moved object, and the old content (if any) has to be copied over to the new page.
All of the functions that were moved were previously without docstrings in SVN, though some had docstrings (that I just now discovered) in the wiki. This may cause some hiccups, I suppose, so I'll be sure to announce when these things get committed to SVN so I know how to help straighten these things out.
Mike
Cheers, Ralf
Mike David Goldsmith wrote: > On Tue, Sep 29, 2009 at 10:55 AM, Michael Droettboom <mdroe@stsci.edu <mailto:mdroe@stsci.edu> > <mailto:mdroe@stsci.edu <mailto:mdroe@stsci.edu>>> wrote: > > 2) Improve documentation > > Every method now has a docstring, and a new page of routines has been > added to the Sphinx tree. > > > Um, where did you do this, 'cause it's not showing up in the doc wiki. > > DG >
> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> > http://mail.scipy.org/mailman/listinfo/numpy-discussion > -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Thanks. There's a bit of a snag getting my SVN permissions, and the doc editor permissions I assume are pending. Once those things are in place, I can move forward on this and hopefully it will be clear what needs to be done.
Mike
David Goldsmith wrote:
On Wed, Sep 30, 2009 at 2:09 PM, Ralf Gommers <ralf.gommers@googlemail.com mailto:ralf.gommers@googlemail.com> wrote:
On Wed, Sep 30, 2009 at 4:37 PM, David Goldsmith <d.l.goldsmith@gmail.com <mailto:d.l.goldsmith@gmail.com>> wrote: So, Ralf (or anyone), how, if at all, should we modify the status of the existing chararray objects/methods in the wiki? Nothing has to be done until *after* Mike has committed his changes to svn. Please see my previous email for what has to happen at that point. Since Mike wrote the new docstrings it would be best if he updated the status of the wiki pages then.
OK; Mike: hopefully it will be clear what you have to do to update the status (it's pretty trivial) but of course don't hesitate to email (you can do so off-list if you prefer) w/ any questions; unfortunately, AFAIK, there's no way to update the status of many docstrings all at once - you'll have to do them each individually (if you like, let me know when you've committed them and I can help - it sounds like there will be a lot); the main "silly" thing to remember is that the option to change the "Review status" only appears if you're logged in. :-)
Assuming you have no problem sharing them with me, Michael, I could add those docstrings you created for the existing methods, They will show up in the wiki when they get committed to svn (presumably within a few days), so this is needless effort for the most part. If there are different changes in the wiki and svn, that will show up in the "merge" page. The ony thing that requires manual effort is if there are changes in the wiki and the object got moved in svn.
And, as above, updating the status in the Wiki. :-)
DG
Cheers, Ralf DG On Wed, Sep 30, 2009 at 7:20 AM, Michael Droettboom <mdroe@stsci.edu <mailto:mdroe@stsci.edu>> wrote: Ralf Gommers wrote: > > > On Wed, Sep 30, 2009 at 9:03 AM, Michael Droettboom <mdroe@stsci.edu <mailto:mdroe@stsci.edu> > <mailto:mdroe@stsci.edu <mailto:mdroe@stsci.edu>>> wrote: > > In the source in my working copy. Is that going to cause problems? I > wasn't sure if it was possible to document methods that didn't yet > exist > in the code in the wiki. > > That is fine. New functions will automatically show up in the wiki. It > would be helpful though if you could mark them ready for review in the > wiki (if they are) after they show up. Could take up to 24 hours for > svn changes to propagate. Thanks. Will do. > > Only if you moved functions around it would be useful if you pinged > Pauli after you committed them. This is a temporary problem, right now > the wiki creates a new page for a moved object, and the old content > (if any) has to be copied over to the new page. All of the functions that were moved were previously without docstrings in SVN, though some had docstrings (that I just now discovered) in the wiki. This may cause some hiccups, I suppose, so I'll be sure to announce when these things get committed to SVN so I know how to help straighten these things out. Mike > > Cheers, > Ralf > > > Mike > > David Goldsmith wrote: > > On Tue, Sep 29, 2009 at 10:55 AM, Michael Droettboom > <mdroe@stsci.edu <mailto:mdroe@stsci.edu> <mailto:mdroe@stsci.edu <mailto:mdroe@stsci.edu>> > > <mailto:mdroe@stsci.edu <mailto:mdroe@stsci.edu> <mailto:mdroe@stsci.edu <mailto:mdroe@stsci.edu>>>> wrote: > > > > 2) Improve documentation > > > > Every method now has a docstring, and a new page of routines > has been > > added to the Sphinx tree. > > > > > > Um, where did you do this, 'cause it's not showing up in the doc > wiki. > > > > DG > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> <mailto:NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org>> > > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > -- > Michael Droettboom > Science Software Branch > Operations and Engineering Division > Space Telescope Science Institute > Operated by AURA for NASA > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> <mailto:NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > ------------------------------------------------------------------------ > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> > http://mail.scipy.org/mailman/listinfo/numpy-discussion > -- Michael Droettboom Science Software Branch Operations and Engineering Division Space Telescope Science Institute Operated by AURA for NASA _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpy-discussion _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpy-discussion _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion