scipy.stats.distributions.erlang - "boring" consensus building in a ticket
Should we restrict the shape parameter to be an integer instead of a float? http://projects.scipy.org/scipy/ticket/1647 Let's ask the users: Does anyone want an exception if the shape parameter is not an integer? Is there a demand or use case for estimating the shape parameter as an integer instead of a float? right now erlang and gamma are essentially the same, as far as I can see Josef
On Sat, May 5, 2012 at 4:20 PM, <josef.pktd@gmail.com> wrote:
Should we restrict the shape parameter to be an integer instead of a float?
http://projects.scipy.org/scipy/ticket/1647
Let's ask the users:
Does anyone want an exception if the shape parameter is not an integer? sorry correction, not an exception, if the parameter is invalid, an nan is returned
Is there a demand or use case for estimating the shape parameter as an integer instead of a float?
right now erlang and gamma are essentially the same, as far as I can see
Josef
From an implementation point of view I think that aliasing Erlang to
Hi, I just looked through the discussion in the ticket. Both sides (1: the scale k should be an int, 2) allow k to be a float) make sense. From my background in queueing I am inclined to say that k should be restricted to the integers as in queueing theory the Erlang-k distribution is used to model some distribution for which k can only be an int. On the other hand, I am unsure whether a user of the Erlang distribution should be protected from filling in a float. In all (?) books on queueing and probability it is written that the Erlang distribution is a special case of the gamma distribution, so users of the Erlang distribution should know this (.... kind of, hopefully). the gamma distribution makes good sense, and I don't believe that the users or the Erlang distribution require to be protected against filling in floats. Perhaps a sentence in the docstring of the Erlang distribution that it is an alias of the gamma distribution, hence does not check on the scale being an int, will prevent some potential misuse. Nicky On 5 May 2012 22:20, <josef.pktd@gmail.com> wrote:
Should we restrict the shape parameter to be an integer instead of a float?
http://projects.scipy.org/scipy/ticket/1647
Let's ask the users:
Does anyone want an exception if the shape parameter is not an integer? Is there a demand or use case for estimating the shape parameter as an integer instead of a float?
right now erlang and gamma are essentially the same, as far as I can see
Josef _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
On Sat, May 5, 2012 at 5:59 PM, nicky van foreest <vanforeest@gmail.com> wrote:
Hi,
I just looked through the discussion in the ticket. Both sides (1: the scale k should be an int, 2) allow k to be a float) make sense. From my background in queueing I am inclined to say that k should be restricted to the integers as in queueing theory the Erlang-k distribution is used to model some distribution for which k can only be an int. On the other hand, I am unsure whether a user of the Erlang distribution should be protected from filling in a float. In all (?) books on queueing and probability it is written that the Erlang distribution is a special case of the gamma distribution, so users of the Erlang distribution should know this (.... kind of, hopefully).
From an implementation point of view I think that aliasing Erlang to the gamma distribution makes good sense, and I don't believe that the users or the Erlang distribution require to be protected against filling in floats. Perhaps a sentence in the docstring of the Erlang distribution that it is an alias of the gamma distribution, hence does not check on the scale being an int, will prevent some potential misuse.
Thanks for the comments. Do you think it would be useful to have fit() restrict to integers? I guess currently nobody uses erlang.fit() because, at least in the example, it doesn't work with default parameters. import numpy as np from scipy import stats #add fitstart to erlang, otherwise it doesn't work stats.erlang._fitstart = stats.gamma._fitstart np.random.seed(876589) rvs = stats.erlang.rvs(5, size=500) for dist in [stats.erlang, stats.gamma]: print '\n', dist.name p0 = dist.fit(rvs) print stats.gamma.nnlf(p0, rvs), p0 print for k in range(10): p = dist.fit(rvs, f0=k) print dist.nnlf(p, rvs), p Josef
Nicky
On 5 May 2012 22:20, <josef.pktd@gmail.com> wrote:
Should we restrict the shape parameter to be an integer instead of a float?
http://projects.scipy.org/scipy/ticket/1647
Let's ask the users:
Does anyone want an exception if the shape parameter is not an integer? Is there a demand or use case for estimating the shape parameter as an integer instead of a float?
right now erlang and gamma are essentially the same, as far as I can see
Josef _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
I don't use the Erlang distribution myself, but would be curious to know how you would go about restricting the scale parameter to being an integer when doing the fit. This kind of restriction sound like it could be useful in a number of different problems, but I can't think of any easy way to do it (other than to use some form of brute force optimisation over the integer parameter). cheers, David ________________________________ From: "josef.pktd@gmail.com" <josef.pktd@gmail.com> To: SciPy Users List <scipy-user@scipy.org> Sent: Monday, 7 May 2012 2:46 AM Subject: Re: [SciPy-User] scipy.stats.distributions.erlang - "boring" consensus building in a ticket On Sat, May 5, 2012 at 5:59 PM, nicky van foreest <vanforeest@gmail.com> wrote:
Hi,
I just looked through the discussion in the ticket. Both sides (1: the scale k should be an int, 2) allow k to be a float) make sense. From my background in queueing I am inclined to say that k should be restricted to the integers as in queueing theory the Erlang-k distribution is used to model some distribution for which k can only be an int. On the other hand, I am unsure whether a user of the Erlang distribution should be protected from filling in a float. In all (?) books on queueing and probability it is written that the Erlang distribution is a special case of the gamma distribution, so users of the Erlang distribution should know this (.... kind of, hopefully).
From an implementation point of view I think that aliasing Erlang to the gamma distribution makes good sense, and I don't believe that the users or the Erlang distribution require to be protected against filling in floats. Perhaps a sentence in the docstring of the Erlang distribution that it is an alias of the gamma distribution, hence does not check on the scale being an int, will prevent some potential misuse.
Thanks for the comments. Do you think it would be useful to have fit() restrict to integers? I guess currently nobody uses erlang.fit() because, at least in the example, it doesn't work with default parameters. import numpy as np from scipy import stats #add fitstart to erlang, otherwise it doesn't work stats.erlang._fitstart = stats.gamma._fitstart np.random.seed(876589) rvs = stats.erlang.rvs(5, size=500) for dist in [stats.erlang, stats.gamma]: print '\n', dist.name p0 = dist.fit(rvs) print stats.gamma.nnlf(p0, rvs), p0 print for k in range(10): p = dist.fit(rvs, f0=k) print dist.nnlf(p, rvs), p Josef
Nicky
On 5 May 2012 22:20, <josef.pktd@gmail.com> wrote:
Should we restrict the shape parameter to be an integer instead of a float?
http://projects.scipy.org/scipy/ticket/1647
Let's ask the users:
Does anyone want an exception if the shape parameter is not an integer? Is there a demand or use case for estimating the shape parameter as an integer instead of a float?
right now erlang and gamma are essentially the same, as far as I can see
Josef _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
On Sun, May 6, 2012 at 6:03 PM, David Baddeley <david_baddeley@yahoo.com.au> wrote:
I don't use the Erlang distribution myself, but would be curious to know how you would go about restricting the scale parameter to being an integer when doing the fit. This kind of restriction sound like it could be useful in a number of different problems, but I can't think of any easy way to do it (other than to use some form of brute force optimisation over the integer parameter).
As in my previous example, I see two possibilities 1) two step optimization, with grid search on the outside and continuous fit for fixed integer parameter on the inside. This works now that constraint fitting is working correctly. 2) run continuous parameter optimization, given that the function is defined on the real line/space, and then go to 1) in the integer neighborhood of the continuous result (other possibility go for openopt or similar with mixed integer programming. I have no idea and never tried) My guess wabout whether 1) or 2) is faster depends on how large the possible range of integers to search in is, and how nice the curvature is to use fmin_bfgs for example. Cheers, Josef
cheers, David
________________________________ From: "josef.pktd@gmail.com" <josef.pktd@gmail.com> To: SciPy Users List <scipy-user@scipy.org> Sent: Monday, 7 May 2012 2:46 AM Subject: Re: [SciPy-User] scipy.stats.distributions.erlang - "boring" consensus building in a ticket
On Sat, May 5, 2012 at 5:59 PM, nicky van foreest <vanforeest@gmail.com> wrote:
Hi,
I just looked through the discussion in the ticket. Both sides (1: the scale k should be an int, 2) allow k to be a float) make sense. From my background in queueing I am inclined to say that k should be restricted to the integers as in queueing theory the Erlang-k distribution is used to model some distribution for which k can only be an int. On the other hand, I am unsure whether a user of the Erlang distribution should be protected from filling in a float. In all (?) books on queueing and probability it is written that the Erlang distribution is a special case of the gamma distribution, so users of the Erlang distribution should know this (.... kind of, hopefully).
From an implementation point of view I think that aliasing Erlang to the gamma distribution makes good sense, and I don't believe that the users or the Erlang distribution require to be protected against filling in floats. Perhaps a sentence in the docstring of the Erlang distribution that it is an alias of the gamma distribution, hence does not check on the scale being an int, will prevent some potential misuse.
Thanks for the comments.
Do you think it would be useful to have fit() restrict to integers?
I guess currently nobody uses erlang.fit() because, at least in the example, it doesn't work with default parameters.
import numpy as np from scipy import stats
#add fitstart to erlang, otherwise it doesn't work stats.erlang._fitstart = stats.gamma._fitstart
np.random.seed(876589) rvs = stats.erlang.rvs(5, size=500)
for dist in [stats.erlang, stats.gamma]: print '\n', dist.name p0 = dist.fit(rvs) print stats.gamma.nnlf(p0, rvs), p0 print
for k in range(10): p = dist.fit(rvs, f0=k) print dist.nnlf(p, rvs), p
Josef
Nicky
On 5 May 2012 22:20, <josef.pktd@gmail.com> wrote:
Should we restrict the shape parameter to be an integer instead of a float?
http://projects.scipy.org/scipy/ticket/1647
Let's ask the users:
Does anyone want an exception if the shape parameter is not an integer? Is there a demand or use case for estimating the shape parameter as an integer instead of a float?
right now erlang and gamma are essentially the same, as far as I can see
Josef _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
Do you think it would be useful to have fit() restrict to integers?
I guess currently nobody uses erlang.fit() because, at least in the example, it doesn't work with default parameters.
import numpy as np from scipy import stats
#add fitstart to erlang, otherwise it doesn't work stats.erlang._fitstart = stats.gamma._fitstart
np.random.seed(876589) rvs = stats.erlang.rvs(5, size=500)
for dist in [stats.erlang, stats.gamma]: print '\n', dist.name p0 = dist.fit(rvs) print stats.gamma.nnlf(p0, rvs), p0 print
for k in range(10): p = dist.fit(rvs, f0=k) print dist.nnlf(p, rvs), p
I think it returning an int is here to be exptected by an Erlang user. Hence, yes. I ran the above code, but got some warnings: Warning: invalid value encountered in subtract I did not pursue the origin of this though.
Josef
Nicky
On 5 May 2012 22:20, <josef.pktd@gmail.com> wrote:
Should we restrict the shape parameter to be an integer instead of a float?
http://projects.scipy.org/scipy/ticket/1647
Let's ask the users:
Does anyone want an exception if the shape parameter is not an integer? Is there a demand or use case for estimating the shape parameter as an integer instead of a float?
right now erlang and gamma are essentially the same, as far as I can see
Josef _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
_______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
participants (3)
-
David Baddeley -
josef.pktd＠gmail.com -
nicky van foreest