[SciPy-Dev] [SciPy-User] log pdf, cdf, etc

Tue Jun 1 21:07:27 EDT 2010

On Tue, Jun 1, 2010 at 5:25 PM, Travis Oliphant <oliphant at enthought.com>wrote:

>
> On Jun 1, 2010, at 8:19 AM, Ralf Gommers wrote:
>
>
>
> On Tue, Jun 1, 2010 at 1:20 PM, Travis Oliphant <oliphant at enthought.com>wrote:
>
>>
>> On May 31, 2010, at 6:39 AM, Ralf Gommers wrote:
>>
>
>
>>  These recent changes are a bit problematic for several reasons:
>> - there are many new methods for distributions without tests.
>>
>> These methods are simple to see and verify.    Which methods specifically
>> are you concerned about?
>>
>
> They're not all simple, for example rv_continuous._reduce_func. Since it
> contains inner function definitions inside an "else" block there's also a
> good chance it's actually broken.
>
> And in principle I'm worried about all of them. The python 2.4/2.5 syntax
> error was caught early, but what if some code you regard as simple is broken
> in a less obvious way on 2.4/2.5? Maybe a user finds it in a release
> candidate, forcing us to build an extra one? Or just after the final
> release?
>
>>
>> - there are no docs for many new private and public methods
>>
>>
>> They are all fairly self explanatory.   But, docs can be added if needed.
>>
>
> For you, and maybe for me too. But for undergraduate students, or Joe in
> accounting who inherited  this random app that's essential for his job? It's
> simple, no public docs without docstrings. And preferably no private ones
> either.
>
> Thanks for fixing all public docs quickly though. You missed just one,
> gamma.fit.
>
>>
>> - invalid syntax: http://projects.scipy.org/scipy/ticket/1186
>>
>>
>> This has been fixed (it was easier to fix the syntax then file the
>> ticket...)  Also to be clear this is only invalid for Python < 2.6 (the
>> comment makes it sound like somehow the changes weren't tested at all).
>>
>> I didn't mean to imply that you were committing code that didn't even work
> for you.
>
>>  - the old rv_continuous doc template was put back in
>>
>>
>> I'm not sure what you mean.   Which change did this?
>>
>
> The first one of your recent commits, r6392. The docstrings for subclasses
> of rv_continuous and rv_discrete are not generated from this template
> anymore, which is why it was removed. Look at line 862 (# generate docstring
> for subclass instances) and below that to see how it works now.
>
> If you're wondering why that changed, the main reasons are (1) to make the
> docstrings conform to the standard, (2) to be able to put useful info in the
> base classes, like "this is how you subclass it: ..." instead of a template,
> and (3) to be able to customize individual distribution docstrings easily.
>
>
>>
>> This, plus Josef saying that he doesn't want to fix the API for some
>> methods yet, makes me want to take it out of the 0.8.x branch. Any
>> objections to that Travis or Josef?
>>
>>
>> I would really like to see these changes go in to 0.8.x.    If Josef feels
>> strongly about the API in the future, we can change it for the next release.
>>   I don't understand what the specific concerns are.
>>
>> No you can't. For API changes we do have a policy, they need deprecation
> first. Which means if we release it like this now, we're stuck with it till
> 0.10 / 1.0.
>
>
>
> In summary, I see quite a few reasons why this shouldn't go in and don't
> see a compelling reason to release it right now. The 0.9 release is
> (tentatively) planned for September, so you don't have to worry that your
> changes sit in trunk unreleased for 1.5 years.
>
>
> As the one doing the work of release manager, you have a lot of latitude in
> making this decision, of course.    The compelling reason to release it
> right now is to get the improved features which nobody has actually voiced
> specific concerns about.
>
>
There have been expressed concerns as to both the design and validation. I
think it should be removed and these changes put into a branch or up on
github until they have been tested and documented. There is no rush, and
really, there is no reason for folks to use code that hasn't been validated
except for testing, and testing can be done using the branch.

> Specifically improvements to the fit method of distribution objects (the
> ability to fix specific parameters of the distribution and vary others in
> the fit) is a very nice-to-have feature.     The API change problem you
> mention is actually an argument for putting it in now (because we *can*
> deprecate it in 0.9 and then have whatever unspecified correct API come out
> in 1.0).    I have not heard that there is real disagreement about the API
> either.
>

No, it is a argument for *not* putting it in now. There is no rush, and
until the code has been looked over and thoroughly tested, there is no
guarantee that either the API is suitable or that the implementation is
correct.

>
> It feels like I've addressed the major reasons you feel it can't go in.
>  The functionality is tested.    There are docstrings.  I just removed the
> rv_continuous doc template.  I really don't know why that was added.   I did
> not make a specific change to include it.  It must have been a merge error.
>
>
We don't know what else might be wrong. Look at what happened with datetime
and all the work that made for David.

> Suggestions about how to give gamma.fit and beta.fit the docstring of it's
> parent would be appreciated.
>
> I don't think a general rule of "no private methods without docstrings" is
> necessarily appropriate, and a bit of an example of going overboard with
> "rules" and "procedures."   Private methods are not meant to be called
> outside of code and should not necessarily have to be documented with
> docstrings.   Every docstring creates more code to maintain and keep
> consistent with the actual code.
>
> One of the great things about Python is that you can read the code itself
> so that it is much closer to self-documenting code
> (close to it but not there --- I like comments and docstrings too).
>
>
Python beyond the trivial is *not* self documenting, no code is self
documenting. There is always a struggle to grasp the larger design and
intent, as well of niggling questions of correctness. All python serves to
do is remove a lot of verbiage by abstracting common objects like lists and
hash tables. That helps, but it is far from all that is needed.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20100601/ce30f486/attachment.html>