[SciPy-Dev] [scipy/scipy] ENH: Add PDF, CDF and parameter estimation for Stable Distributions (#7374)

Tue Apr 10 11:34:21 EDT 2018

On 10 April 2018 at 12:34, Andrea Karlova <andrea.karlova at gmail.com> wrote:

> On 8 April 2018 at 18:05, Blair Azzopardi <blairuk at gmail.com> wrote:
>
>> > On 8 April 2018 at 12:49, Andrea Karlova <andrea.karlova at gmail.com>
>> wrote:
>> >> On 8 April 2018 at 05:21, Blair Azzopardi <blairuk at gmail.com> wrote:
>> >>> On 8 April 2018 at 01:49, Andrea Karlova <andrea.karlova at gmail.com>
>> wrote:
>> >>>> On 28 November 2017 at 18:33, Blair Azzopardi <blairuk at gmail.com>
>> wrote:
>> >>>>> On Tue, 28 Nov 2017, 16:01 Andrea Karlova, <
>> andrea.karlova at gmail.com> wrote:
>> >>>>>> On 28 November 2017 at 13:17, Blair Azzopardi <blairuk at gmail.com>
>> wrote:
>> >>>>>>> ---------- Forwarded message ---------
>> >>>>>>> From: An <notifications at github.com>
>> >>>>>>> Date: Tue, 28 Nov 2017, 10:00
>> >>>>>>> Subject: Re: [scipy/scipy] ENH: Add PDF, CDF and parameter
>> estimation for Stable Distributions (#7374)
>> >>>>>>> To: scipy/scipy <scipy at noreply.github.com>
>> >>>>>>>
>> >>>>>>> Blair, is there a way to have a chat via email?
>> >>>>>>>
>> >>>>>>> On 24 November 2017 at 20:38, Blair Azzopardi <
>> notifications at github.com>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>> Hi @an81 <https://github.com/an81>. Thank you for the 550+ page
>> book.
>> >>>>>>> Please can you be a bit more specific? Some sample code goes a
>> long way
>> >>>>>>> too. Also can you perhaps test the existing code and highlight
>> where the
>> >>>>>>> Gibbs effect might be more prominent? eg low alpha etc; perhaps
>> this can be
>> >>>>>>> just documented with a recommendation that users use quad in
>> these cases
>> >>>>>>> (already in code). This is until better implementation is
>> available.
>> >>>>>>> —
>> >>>>>>> You are receiving this because you were mentioned.
>> >>>>>>> Reply to this email directly, view it on GitHub
>> >>>>>>> <https://github.com/scipy/scipy/pull/7374#issuecomment-346893569>,
>> or mute
>> >>>>>>> the thread
>> >>>>>>> <https://github.com/notifications/unsubscribe-auth/AIfE8VNjH
>> dqANvYfUG8Gg6feKb5np_kLks5s5ylhgaJpZM4NQBiP>
>> >>>>>>
>> >>>>>> Hi Andrea
>> >>>>>>
>> >>>>>> I hope you're well and yes no problem talking chatting via email.
>> Actually makes sense.
>> >>>>>>
>> >>>>>> Also, I hope you don't mind me messaging you via an email address
>> found in a previous comment.
>> >>>>>>
>> >>>>>> I took a look at that book you linked to but unfortunately I don't
>> have enough time to process it currently. I will read it in time mind you.
>> I've found a shorter paper that might offer similar suggestions to yours:
>> >>>>>>
>> >>>>>> http://prac.im.pwr.edu.pl/~hugo/publ/SFB2005-008_Borak_Haerd
>> le_Weron.pdf
>> >>>>>>
>> >>>>>> Although I'm not 100% sure as it doesn't mention mejer g
>> functions. What are your thoughts?
>> >>>>>>
>> >>>>>> Kind regards
>> >>>>>> Blair
>> >>>>>
>> >>>>> Hi Blair,
>> >>>>>
>> >>>>> thats great.
>> >>>>> Thx for your email.
>> >>>>>
>> >>>>> I ll give you access to my Dropbox folder where I have materials
>> relevant to stable laws.
>> >>>>> For working with stable laws I found really useful to understand
>> >>>>> and actively switch between different parametrizations of the
>> characteristic exponent.
>> >>>>> Zolotarev and others would have polenty of parametrization and each
>> of them is helpful
>> >>>>> for different task.
>> >>>>>
>> >>>>> I guess I can share with you my python  code on the github,
>> >>>>> you can contribute to it if you would feel like so
>> >>>>> and then we can just plug it into scipy lib.
>> >>>>>
>> >>>>> Is it ok to use this gmail account for sending you invitation to
>> Dropbox?
>> >>>>>
>> >>>>> Thx,
>> >>>>> Kind regards,
>> >>>>>
>> >>>>> Andrea
>> >>>>
>> >>>> Hi Andrea
>> >>>>
>> >>>> Yes, please do share with this email address.
>> >>>>
>> >>>> Is it possible you could commit your changes to the existing PR I've
>> already set up? I believe this is possible by forking my repo (script fork)
>> and committing to that. The PR includes parameter estimation and some
>> general framework changes around this distribution too. Also I feel there
>> are still use cases where it's useful keep fft method. I've added you a
>> collaborator on my fork.
>> >>>>
>> >>>> https://github.com/bsdz/scipy
>> >>>>
>> >>>> Kind regards
>> >>>> Blair
>> >>>
>> >>> Hi Blair,
>> >>>
>> >>> hope you are well.
>> >>> I was looking into your code on stable laws in scipy.
>> >>> I ll be running a group of 4 people at hackaton which is  organized
>> by AHL in 2 weeks time
>> >>> and I am planning to revise and add more code into stable laws
>> implementation.
>> >>>
>> >>> I was wondering if you can give me some quick update on what is done
>> so far
>> >>> and what are urgent issues at this point according to your opinion?
>> >>> I have some ideas what I would like add, also the documentation page
>> needs to be written,
>> >>> but just a quick check on what methods you implemented for
>> calculating of
>> >>> pdfs, cdfs, and parameters estimates and how it went?
>> >>>
>> >>> Thanks a lot.
>> >>> Kind regards,
>> >>> Andrea
>> >>>
>> >>
>> >> Hi Andrea
>> >>
>> >> Thanks for your email. I'm cc-ing this email along with out previous
>> correspondence to the scipy-dev mailing list (as bottom post) . Please
>> direct all your future messages here.
>> >>
>> >> It's good to hear you plan to improve my PR (coincidentally at my old
>> employer Man, although at the time they weren't interested in Stable laws).
>> >>
>> >> I did implement your suggestions to use Zolotarev's formulations. All
>> the code is under https://github.com/scipy/scipy/pull/7374 as you are
>> aware.
>> >>
>> >> You could perhaps look at improving documentation, adding more /
>> improving tests and any optimisations you can think of.
>> >>
>> >> Either I can re-add you as a collaborator to my repo (I removed you
>> when I hadn't heard back from you previously) or you can email me a patch
>> and I'll integrate it into PR 7374.
>> >>
>> >> Thanks
>> >> Blair
>> >
>> > Hi Blair,
>> >
>> > yes, if you could add me to your repository,
>> > so I can look into what is done and start to test your code,
>> > so that I can look into some patches, that would be great.
>>
>> Hi Andrea
>>
>> Actually, just to be cautious. Could you fork my scipy repo (it has all
>> the changes there) and then create a PR into that repo? I can then merge it
>> and it should push upstream into my main PR.
>>
>
> Sure, no worries.
>
>
>>
>> > How does your code performs in neighbourhood of alpha = 1?
>> > How does it perform when the asymetry parameter beta gets close to its
>> boundary values?
>>
>> Please run the code and check. My testing took samples from Nolan's
>> public domain executables and tested against those values. Although I
>> tested against a range of parameters I might not have tested those ranges
>> specifically. It's all in the unit tests in the PR, see test class
>> TestLevyStable. Extending the testing here would be helpful.
>>
>
> Ok.
>
>
>>
>> > Do you have somewhere in Latex notes on your fft implementation,
>> > with some theoretical investigation of the performance of the method?
>>
>> This is all in the code documentation along with references.
>>
>
> Ok, so the answer to my question is no and there is only available info in
> the comments in the code, right?
>

It's in the code, please look at function _pdf_from_cf_with_fft(), it has a
docstring that refers to "[MS]". The full reference is then in the main
docstring along with page numbers.

https://github.com/scipy/scipy/pull/7374/files#diff-6e4287784ff3e7a5c6d89d442bea3fbbR3683-R3694

>
>> > I am planning to write a research paper about the implementation in the
>> Scipy, which I ll put on my ArXiv,
>> > and to which I ll refer in the documentation, so that if people are
>> interested in the performance, they can look into paper.
>> > So I ll be listing the co-authors of this piece of code there.
>> > If there are other people who worked on the stable laws implementation
>> in Scipy, it would be nice if I can have the list of them.
>> > Also if you have some note on fft,  I can join it into my research note
>> and add you as  a coauthor of the note,
>> > depends upon you.
>>
>> So far, I'm the sole author of this PR. Again, all papers and references
>> are in the PR. Please please do look at the code.
>>
> Ok, done. Thanks.
>
>
>>
>> >
>> > Kind regards,
>> > Andrea
>>
>> Just some additional points, you need to subscribe to the scipy-dev
>> mailing list for your messages to get through. Also, the mailing list uses
>> bottom posting for email messages; that means, all replies are bottom of
>> original message (not the top.)
>>
>>
> Sure.
>

Btw, it seems your messages aren't getting through to scip-dev mailing
list. You can subscribe here:

https://mail.python.org/mailman/listinfo/scipy-dev

>
>
>> Look forward to your contribution - thanks!
>>
>
> Me too :)
>
>>
>> Blair
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180410/d9282dbf/attachment-0001.html>