[SciPy-Dev] Review of PR 8293 - sampling of random variates

Phillip Feldman phillip.m.feldman at gmail.com
Sun Aug 5 03:13:34 EDT 2018 

Devroye's book is excellent.  Thanks for reminding me of it!

On Sat, Aug 4, 2018 at 11:59 PM, Christoph Baumgarten <
christoph.baumgarten at gmail.com> wrote:

>
> A good source on the ratio of uniforms method is the book by Devroye:
> "non-uniform random variate generation" or the original paper "Computer
> Computer Generation of Random Variables using the ratio of uniform
> deviates" by kinderman / monahan.
>
> <scipy-dev-request at python.org> schrieb am Sa., 4. Aug. 2018, 18:03:
>
>> Send SciPy-Dev mailing list submissions to
>>         scipy-dev at python.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>         https://mail.python.org/mailman/listinfo/scipy-dev
>> or, via email, send a message with subject or body 'help' to
>>         scipy-dev-request at python.org
>>
>> You can reach the person managing the list at
>>         scipy-dev-owner at python.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of SciPy-Dev digest..."
>>
>>
>> Today's Topics:
>>
>>    1. Re: Review of PR 8293 - sampling of random variates (Neal Becker)
>>    2. Re: Backwards Compatibility for low level LAPACK routines
>>       (Ralf Gommers)
>>    3. Re: Backwards Compatibility for low level LAPACK routines
>>       (Pauli Virtanen)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Sat, 4 Aug 2018 07:44:06 -0400
>> From: Neal Becker <ndbecker2 at gmail.com>
>> To: SciPy Developers List <scipy-dev at python.org>
>> Subject: Re: [SciPy-Dev] Review of PR 8293 - sampling of random
>>         variates
>> Message-ID:
>>         <CAG3t+pF_5GMXGDJUJF-Z8n2iRHvTSpwG6_b4y1dMWSqwKS_
>> 29w at mail.gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>>
>> Don't know about this particular code, but unuran is pretty comprehensive,
>> and there's a good book from the author.
>>
>> On Fri, Aug 3, 2018, 5:37 PM Phillip Feldman <phillip.m.feldman at gmail.com
>> >
>> wrote:
>>
>> > The ability to sample random variates from a distribution where only the
>> > density is available seems quite useful.  Is there a reference that
>> > describes the method?
>> >
>> > Phillip
>> >
>> > On Thu, Aug 2, 2018 at 11:43 PM, Christoph Baumgarten <
>> > christoph.baumgarten at gmail.com> wrote:
>> >
>> >> PR 8293 introduces a method to sample random variates for more complex
>> >> distributions to scipy.stats (such as hyperbolic distributions and the
>> >> generalized inverse gaussian (see PR 8681)). I think the PR is in good
>> >> shape, it would be great if it could be merged soon since i could then
>> >> continue to work on adding new distributions. The PR has been open for
>> 7
>> >> months now. If someone could continue the review, that would bei great.
>> >> Thanks!
>> >>
>> >> _______________________________________________
>> >> SciPy-Dev mailing list
>> >> SciPy-Dev at python.org
>> >> https://mail.python.org/mailman/listinfo/scipy-dev
>> >>
>> >>
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at python.org
>> > https://mail.python.org/mailman/listinfo/scipy-dev
>> >
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <http://mail.python.org/pipermail/scipy-dev/
>> attachments/20180804/73dd5df9/attachment-0001.html>
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Sat, 4 Aug 2018 06:06:22 -0700
>> From: Ralf Gommers <ralf.gommers at gmail.com>
>> To: SciPy Developers List <scipy-dev at python.org>
>> Subject: Re: [SciPy-Dev] Backwards Compatibility for low level LAPACK
>>         routines
>> Message-ID:
>>         <CABL7CQgmuAF2V0Pyrtt5qEBPE9M1ostJL54fx8YVVTZbgRF7KQ at mail.
>> gmail.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> On Sat, Aug 4, 2018 at 12:49 AM, Ilhan Polat <ilhanpolat at gmail.com>
>> wrote:
>>
>> > It will gain a dgeqrf_lwork function as usual to return the necessary
>> > workspace size as
>> >
>> > lwork, info = dgeqrf_lwork(m,n)
>> >
>> > Then the "work" variable will be removed from dgeqrf signature and will
>> > be made hidden.
>> >
>> > For example for the previous example I gave before, the optimal size is
>> > 12800 and work array is returning an 12800-long array for a 400-400
>> array
>> > computation.
>> >
>>
>> Ah okay. Then the alternative is to just leave the work parameter, ignore
>> it in the code if it's passed in (or give a warning/error) and document it
>> as not being used. Right?
>>
>> If you're removing "work" from both the signature and the return value,
>> that's a bigger change indeed, that can't be handled well that way. I'm
>> not
>> 100% sure, but I think I agree that a backwards incompatible change here
>> will be better than introducing a bunch of new functions with worse names.
>>
>> We could introduce a Python wrapper for these to give a proper
>> FutureWarning first.
>>
>> Cheers,
>> Ralf
>>
>>
>>
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Sat, Aug 4, 2018 at 3:25 AM, Ralf Gommers <ralf.gommers at gmail.com>
>> > wrote:
>> >
>> >>
>> >>
>> >> On Thu, Aug 2, 2018 at 11:37 AM, Ilhan Polat <ilhanpolat at gmail.com>
>> >> wrote:
>> >>
>> >>> Due their historical evolution, there are certain LAPACK wrappers that
>> >>> are not standardized. Some work with minimum lwork variables instead
>> of
>> >>> their optimal values. Also these routines often return quite big
>> arrays
>> >>> during the lwork queries, to demonstrate :
>> >>>
>> >>> import scipy.linalg as la
>> >>> la.lapack.dgeqrf(a=np.random.rand(400,400), lwork=-1)
>> >>>
>> >>> is a workspace size query (via lwork=-1). The current default size is
>> >>> "3*a.shape[0] + 1" hence 1201. However the optimal workspace size is
>> 12800
>> >>> on my machine. Therefore the mismatch is sometimes quite dramatic
>> >>> especially in some other routines. Notice also that to obtain this
>> number
>> >>> the routine actually returns a 400-long array tau and requires the
>> input
>> >>> matrix to be transferred back and forth. Moreover, they can't be
>> handled
>> >>> via scipy.linalg.lapack._compute_lwork function.
>> >>>
>> >>> There are a few routines like this and I feel like they should be
>> fixed
>> >>> and I'm willing to. However this means that their output signature is
>> going
>> >>> to change which imply backwards compatibility breaks.
>> >>>
>> >>
>> >> What would the output change to? Currently it returns:
>> >>
>> >>     qr : rank-2 array('d') with bounds (m,n) and a storage
>> >>     tau : rank-1 array('d') with bounds (MIN(m,n))
>> >>     work : rank-1 array('d') with bounds (MAX(lwork,1))
>> >>     info : int
>> >>
>> >> Ralf
>> >>
>> >>
>> >>> I tried to see whether we could deprecate them with new wrappers with
>> >>> modified names, but to be honest, that would create too many
>> duplicates. On
>> >>> the other hand I don't have a feeling of how much break this would
>> mean out
>> >>> there in the wild.
>> >>>
>> >>> Is this break an acceptable one or not? (well, none is acceptable
>> >>> preferably, but in despair...)
>> >>>
>> >>> Any other alternatives, thoughts are most welcome.
>> >>>
>> >>> best,
>> >>> ilhan
>> >>>
>> >>> _______________________________________________
>> >>> SciPy-Dev mailing list
>> >>> SciPy-Dev at python.org
>> >>> https://mail.python.org/mailman/listinfo/scipy-dev
>> >>>
>> >>>
>> >>
>> >> _______________________________________________
>> >> SciPy-Dev mailing list
>> >> SciPy-Dev at python.org
>> >> https://mail.python.org/mailman/listinfo/scipy-dev
>> >>
>> >>
>> >
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at python.org
>> > https://mail.python.org/mailman/listinfo/scipy-dev
>> >
>> >
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <http://mail.python.org/pipermail/scipy-dev/
>> attachments/20180804/d1e8615c/attachment-0001.html>
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Sat, 04 Aug 2018 16:00:36 +0200
>> From: Pauli Virtanen <pav at iki.fi>
>> To: scipy-dev at python.org
>> Subject: Re: [SciPy-Dev] Backwards Compatibility for low level LAPACK
>>         routines
>> Message-ID: <5b464122e6ce8e5a7959f325e534f8d2543801d5.camel at iki.fi>
>> Content-Type: text/plain; charset="UTF-8"
>>
>> la, 2018-08-04 kello 06:06 -0700, Ralf Gommers kirjoitti:
>> [clip]
>> > Ah okay. Then the alternative is to just leave the work parameter,
>> > ignore
>> > it in the code if it's passed in (or give a warning/error) and
>> > document it
>> > as not being used. Right?
>> >
>> > If you're removing "work" from both the signature and the return
>> > value,
>> > that's a bigger change indeed, that can't be handled well that way.
>> > I'm not
>> > 100% sure, but I think I agree that a backwards incompatible change
>> > here
>> > will be better than introducing a bunch of new functions with worse
>> > names.
>> >
>> > We could introduce a Python wrapper for these to give a proper
>> > FutureWarning first.
>>
>> Or, perhaps you can leave the `work` return variable in, but make it an
>> 1-element array? Its value can be filled in from the `callstatement`,
>> cf eg
>> https://github.com/scipy/scipy/blob/master/scipy/
>> linalg/flapack_gen.pyf.src#L89
>> The actual work array is then made an intent(hide,cache) variable.
>>
>>         Pauli
>>
>>
>>
>>
>> ------------------------------
>>
>> Subject: Digest Footer
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>>
>> ------------------------------
>>
>> End of SciPy-Dev Digest, Vol 178, Issue 6
>> *****************************************
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180805/c59efed2/attachment-0001.html>

More information about the SciPy-Dev mailing list