<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 17, 2020 at 9:44 AM Romain Jacob <<a href="mailto:jacobr@ethz.ch">jacobr@ethz.ch</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hello everyone,</p>
<p>I've submitted the PR adding support for non-parametric
confidence intervals for quantiles (<a href="https://github.com/scipy/scipy/pull/12680" target="_blank">https://github.com/scipy/scipy/pull/12680</a>).
There has been quite some comments made already, which I fixed
appropriately I believe<br>
</p>
<p>Will be happy to get some more feedback or see the PR merged :-)<br>
</p>
<p>Note: the last commit has a CI failing apparently due to a file
change in `scipy/sparse/linalg/` which is completely unrelated.
I'm not sure how to go about this... ?<br></p></div></blockquote><div>If it's clearly unrelated, you can just ignore it. Or add a comment "the only CI failure is in sparse.linalg and unrelated to this PR". Then the reviewer can just go ahead and merge if everything else looks good - CI doesn't have to be green.</div><div><br></div><div>Cheers,<br></div><div>Ralf</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p>
</p>
Cheers,<br>
-- <br>
Romain<br>
<div><br>
</div>
<div>On 15/06/2020 08:27, Romain Jacob
wrote:<br>
</div>
<blockquote type="cite">
<div>On 13/06/2020 20:54, <a href="mailto:josef.pktd@gmail.com" target="_blank">josef.pktd@gmail.com</a>
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">On Fri, Jun 12, 2020 at 11:29 AM <<a href="mailto:josef.pktd@gmail.com" target="_blank">josef.pktd@gmail.com</a>>
wrote:<br>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">On Fri, Jun 12, 2020 at 1:58 AM Romain
Jacob <<a href="mailto:jacobr@ethz.ch" target="_blank">jacobr@ethz.ch</a>>
wrote:<br>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>On 11/06/2020 20:54, Warren Weckesser wrote:<br>
</div>
<blockquote type="cite">
<pre>On 6/11/20, <a href="mailto:josef.pktd@gmail.com" target="_blank">josef.pktd@gmail.com</a> <a href="mailto:josef.pktd@gmail.com" target="_blank"><josef.pktd@gmail.com></a> wrote:
</pre>
<blockquote type="cite">
<pre>I think it would make a good and useful addition and fit into scipy.stats.
There are no pure confint functions yet, AFAIR.
</pre>
</blockquote>
<pre>I agree with Josef and Matt, this looks like it would be a nice
addition to SciPy. At the moment, I'm not sure what the API should
look like. Romain, is the work that you've already done available
online somewhere?
Warren
</pre>
</blockquote>
<p>Yes, I have some functional implementation
available here: <a href="https://github.com/TriScale-Anon/triscale/blob/master/helpers.py#L397" target="_blank">https://github.com/TriScale-Anon/triscale/blob/master/helpers.py#L397</a></p>
</div>
</blockquote>
<div><br>
</div>
<div>An implementation detail: </div>
<div>binom has cdf and ppf functions</div>
<div>My guess, not verified, is that we can just use
binom.interval</div>
<div><br>
</div>
<div>(at least I used those for similar cases)</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I found my version again</div>
<div><a href="https://github.com/statsmodels/statsmodels/issues/6562#issuecomment-592769480" target="_blank">https://github.com/statsmodels/statsmodels/issues/6562#issuecomment-592769480</a> </div>
<div><br>
</div>
<div>I guess that's the same for two sided confint as the
references. </div>
<div>It doesn't have interpolation if that could be applied
in this case. <br>
</div>
<div><br>
</div>
</div>
</div>
</blockquote>
<p>I don't entirely follow what you mean here: that the building
of the probabilities in these two lines( <a href="https://github.com/TriScale-Anon/triscale/blob/master/helpers.py#L438" target="_blank">https://github.com/TriScale-Anon/triscale/blob/master/helpers.py#L438</a>
and L439) can be built directly form binom without np.cumsum?
That definitely correct (I actually have code also doing that
somewhere).</p>
<p>I did not know about the `interval` method. That's sound
interesting indeed, but it's not 100% clear to me how the
uniqueness problem is handled. I looked for the implementation
of the method but couldn't find it in `binom`... I'm looking in
the wrong place?</p>
<p>Cheers,<br>
-- <br>
Romain<br>
</p>
<p><br>
</p>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div>This will eventually end up in statsmodels, but I don't
know yet where. That's not a reason not to add it to
scipy.stats.</div>
<div><br>
</div>
<div>Josef</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
</div>
<div>Josef</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>There is quite some work to be done on
formatting and documentation to comply with the
SciPy standards, but functionally it's already
there (and as you'll see, the method is quite
simple).</p>
<p>Cheers,<br>
-- <br>
Romain<br>
</p>
<blockquote type="cite">
<blockquote type="cite">
<pre>I recently wrote a function for the confidence interval for the median,
mainly because I ran into the formulas that were easy to code.
related open issue: how do we get confidence intervals for QQ-plot.
aside: I don't like "percent", I prefer quantiles in [0, 1]. See discussion
a while ago in numpy.
Josef
On Thu, Jun 11, 2020 at 1:01 PM Matt Haberland <a href="mailto:mhaberla@calpoly.edu" target="_blank"><mhaberla@calpoly.edu></a>
wrote:
</pre>
<blockquote type="cite">
<pre>OK, we should let our statistics experts weigh in on this. (I'm not
actually one of them.)
On Wed, Jun 10, 2020 at 10:46 PM Romain Jacob <a href="mailto:jacobr@ethz.ch" target="_blank"><jacobr@ethz.ch></a> wrote:
</pre>
<blockquote type="cite">
<pre>I think a dedicated function makes more sense. This function takes as
input an array, a percentile and a confidence level, and returns the
corresponding one-sided confidence intervals.
I quickly looked at the list of existing functions in scipy.stats but
did
not see any function in "summary statistics" that does similar things. So
I
would go for a new function.
On 10/06/2020 20:38, Matt Haberland wrote:
Where do you envision this living in SciPy? In its own function, or
added
functionality to other functions e.g. scipy.stats.percentileofscore
<a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.percentileofscore.html#scipy.stats.percentileofscore" target="_blank"><https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.percentileofscore.html#scipy.stats.percentileofscore></a>
?
On Tue, Jun 9, 2020 at 11:12 PM Romain Jacob <a href="mailto:jacobr@ethz.ch" target="_blank"><jacobr@ethz.ch></a> wrote:
</pre>
<blockquote type="cite">
<pre>On 09/06/2020 20:18, Matt Haberland wrote:
Yes, I think we would be interested in confidence intervals, but I
think
the algorithm should be very well standard/cited, even if it's not the
best/most modern.
Yes definitely! We did not invented the method I am referring to, it a
long-known approach (first proposed by Thompson in 1936 [1], extended
later
and commonly found in textbooks, eg [2,3]). This method is very simple,
quite powerful, yet it has been largely overlooked in many scientific
fields. I found no available implementation to facilitate its use (at
least
not in Python, there may be something in R, I have not looked).
[1] <a href="https://www.jstor.org/stable/2957563" target="_blank">https://www.jstor.org/stable/2957563</a>
[2] <a href="http://doi.org/10.1002/0471722162.ch7" target="_blank">doi.org/10.1002/0471722162.ch7</a>
[3] <a href="https://perfeval.epfl.ch/" target="_blank">https://perfeval.epfl.ch/</a>
@WarrenWeckesser and I had planned to work on confidence intervals for
the test statistics returned by our statistical tests
<a href="https://docs.scipy.org/doc/scipy/reference/stats.html#statistical-tests" target="_blank"><https://docs.scipy.org/doc/scipy/reference/stats.html#statistical-tests></a>.
That is also definitely interesting, although I am not myself an expert
in that area. I am glad to see that the complete list contains some
non-parametric tests :-)
Cheers,
--
Romain
On Mon, Jun 8, 2020 at 2:11 AM Romain Jacob <a href="mailto:jacobr@ethz.ch" target="_blank"><jacobr@ethz.ch></a> wrote:
</pre>
<blockquote type="cite">
<pre>Hello everyone,
I have been working for some time on the implementation of
non-parametric methods to compute confidence intervals for
percentiles.
There are some very interesting results in the literature (see e.g. a
nice
pitch in [1]) which I think it would be great to add to SciPy to make
them
more readily available. It also seems to be rather in line with
"recent"
discussions of the roadmap for scipy.stats [2].
I would be interested in contributing this. What do you think?
Cheers,
--
Romain
[1] <a href="https://ieeexplore.ieee.org/document/6841797" target="_blank">https://ieeexplore.ieee.org/document/6841797</a>
[2] <a href="https://github.com/scipy/scipy/issues/10577" target="_blank">https://github.com/scipy/scipy/issues/10577</a>
--
Romain Jacob
Postdoctoral Researcher
ETH Zurich - Computer Engineering and Networks Laboratory
<a href="http://www.romainjacob.net" target="_blank">www.romainjacob.net</a>
@RJacobPartner <a href="https://twitter.com/RJacobPartner" target="_blank"><https://twitter.com/RJacobPartner></a>
Gloriastrasse 35, ETZ G75
8092 Zurich
+41 7 68 16 88 22
_______________________________________________
SciPy-Dev mailing list
<a href="mailto:SciPy-Dev@python.org" target="_blank">SciPy-Dev@python.org</a>
<a href="https://mail.python.org/mailman/listinfo/scipy-dev" target="_blank">https://mail.python.org/mailman/listinfo/scipy-dev</a>
</pre>
</blockquote>
<pre>--
Matt Haberland
Assistant Professor
BioResource and Agricultural Engineering
08A-3K, Cal Poly
_______________________________________________
SciPy-Dev mailing
<a href="mailto:listSciPy-Dev@python.orghttps://mail.python.org/mailman/listinfo/scipy-dev" target="_blank">listSciPy-Dev@python.orghttps://mail.python.org/mailman/listinfo/scipy-dev</a>
--
Romain Jacob
Postdoctoral Researcher
ETH Zurich - Computer Engineering and Networks Laboratory
<a href="http://www.romainjacob.net" target="_blank">www.romainjacob.net</a>
@RJacobPartner <a href="https://twitter.com/RJacobPartner" target="_blank"><https://twitter.com/RJacobPartner></a>
Gloriastrasse 35, ETZ G75
8092 Zurich
+41 7 68 16 88 22
_______________________________________________
SciPy-Dev mailing list
<a href="mailto:SciPy-Dev@python.org" target="_blank">SciPy-Dev@python.org</a>
<a href="https://mail.python.org/mailman/listinfo/scipy-dev" target="_blank">https://mail.python.org/mailman/listinfo/scipy-dev</a>
</pre>
</blockquote>
<pre>--
Matt Haberland
Assistant Professor
BioResource and Agricultural Engineering
08A-3K, Cal Poly
_______________________________________________
SciPy-Dev mailing
<a href="mailto:listSciPy-Dev@python.orghttps://mail.python.org/mailman/listinfo/scipy-dev" target="_blank">listSciPy-Dev@python.orghttps://mail.python.org/mailman/listinfo/scipy-dev</a>
_______________________________________________
SciPy-Dev mailing list
<a href="mailto:SciPy-Dev@python.org" target="_blank">SciPy-Dev@python.org</a>
<a href="https://mail.python.org/mailman/listinfo/scipy-dev" target="_blank">https://mail.python.org/mailman/listinfo/scipy-dev</a>
</pre>
</blockquote>
<pre>--
Matt Haberland
Assistant Professor
BioResource and Agricultural Engineering
08A-3K, Cal Poly
_______________________________________________
SciPy-Dev mailing list
<a href="mailto:SciPy-Dev@python.org" target="_blank">SciPy-Dev@python.org</a>
<a href="https://mail.python.org/mailman/listinfo/scipy-dev" target="_blank">https://mail.python.org/mailman/listinfo/scipy-dev</a>
</pre>
</blockquote>
</blockquote>
<pre>_______________________________________________
SciPy-Dev mailing list
<a href="mailto:SciPy-Dev@python.org" target="_blank">SciPy-Dev@python.org</a>
<a href="https://mail.python.org/mailman/listinfo/scipy-dev" target="_blank">https://mail.python.org/mailman/listinfo/scipy-dev</a>
</pre>
</blockquote>
<div>-- <br>
<div style="font-family:Arial,sans-serif;margin-left:0px">
<div style="color:rgb(31,64,122);font-size:0.9em;margin-bottom:0.2em;font-weight:bold">Romain
Jacob</div>
<div style="color:rgb(31,64,122);font-size:0.9em;margin-bottom:0.15em">Postdoctoral
Researcher</div>
<div style="color:rgb(31,64,122);font-size:0.75em;margin-bottom:0.5em">ETH
Zurich - Computer Engineering and Networks
Laboratory </div>
<div> <a href="https://www.romainjacob.net/" style="color:rgb(31,64,122);font-size:0.9em" target="_blank">www.romainjacob.net</a>
<br>
<a href="https://twitter.com/RJacobPartner" style="color:rgb(31,64,122);font-size:0.9em" target="_blank">@RJacobPartner</a>
</div>
<div style="font-size:0.9em;margin-top:0.5em">
Gloriastrasse 35, ETZ G75<br>
8092 Zurich <br>
+41 7 68 16 88 22 </div>
</div>
</div>
</div>
_______________________________________________<br>
SciPy-Dev mailing list<br>
<a href="mailto:SciPy-Dev@python.org" target="_blank">SciPy-Dev@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scipy-dev" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scipy-dev</a><br>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
SciPy-Dev mailing list
<a href="mailto:SciPy-Dev@python.org" target="_blank">SciPy-Dev@python.org</a>
<a href="https://mail.python.org/mailman/listinfo/scipy-dev" target="_blank">https://mail.python.org/mailman/listinfo/scipy-dev</a>
</pre>
</blockquote>
<div>-- <br>
<div style="font-family:"Arial",sans-serif;margin-left:0px">
<div style="color:rgb(31,64,122);font-size:0.9em;margin-bottom:0.2em;font-weight:bold">Romain Jacob</div>
<div style="color:rgb(31,64,122);font-size:0.9em;margin-bottom:0.15em">Postdoctoral Researcher</div>
<div style="color:rgb(31,64,122);font-size:0.75em;margin-bottom:0.5em">ETH Zurich - Computer Engineering and Networks
Laboratory </div>
<div> <a href="https://www.romainjacob.net/" style="color:rgb(31,64,122);font-size:0.9em" target="_blank">www.romainjacob.net</a>
<br>
<a href="https://twitter.com/RJacobPartner" style="color:rgb(31,64,122);font-size:0.9em" target="_blank">@RJacobPartner</a>
</div>
<div style="font-size:0.9em;margin-top:0.5em">
Gloriastrasse 35, ETZ G75<br>
8092 Zurich <br>
+41 7 68 16 88 22 </div>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
SciPy-Dev mailing list
<a href="mailto:SciPy-Dev@python.org" target="_blank">SciPy-Dev@python.org</a>
<a href="https://mail.python.org/mailman/listinfo/scipy-dev" target="_blank">https://mail.python.org/mailman/listinfo/scipy-dev</a>
</pre>
</blockquote>
<div>-- <br>
<div style="font-family:"Arial",sans-serif;margin-left:0px">
<div style="color:rgb(31,64,122);font-size:0.9em;margin-bottom:0.2em;font-weight:bold">Romain Jacob</div>
<div style="color:rgb(31,64,122);font-size:0.9em;margin-bottom:0.15em">Postdoctoral Researcher</div>
<div style="color:rgb(31,64,122);font-size:0.75em;margin-bottom:0.5em">ETH Zurich - Computer Engineering and Networks
Laboratory </div>
<div> <a href="https://www.romainjacob.net/" style="color:rgb(31,64,122);font-size:0.9em" target="_blank">www.romainjacob.net</a>
<br>
<a href="https://twitter.com/RJacobPartner" style="color:rgb(31,64,122);font-size:0.9em" target="_blank">@RJacobPartner</a>
</div>
<div style="font-size:0.9em;margin-top:0.5em"> Gloriastrasse
35, ETZ G75<br>
8092 Zurich <br>
+41 7 68 16 88 22 </div>
</div>
</div>
</div>
_______________________________________________<br>
SciPy-Dev mailing list<br>
<a href="mailto:SciPy-Dev@python.org" target="_blank">SciPy-Dev@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scipy-dev" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scipy-dev</a><br>
</blockquote></div></div>