[Distutils] PyPI Download Counts

[holger krekel]

On Sun, Jun 09, 2013 at 01:57 +0300, anatoly techtonik wrote:
> On Thu, May 30, 2013 at 6:56 PM, Donald Stufft <donald at stufft.io> wrote:
> 
> >
> > On May 30, 2013, at 8:05 AM, Jim Fulton <jim at zope.com> wrote:
> >
> > On Mon, May 27, 2013 at 3:27 AM, holger krekel <holger at merlinux.eu> wrote:
> >
> > Hi Donald,
> >
> > On Sun, May 26, 2013 at 20:08 -0400, Donald Stufft wrote:
> >
> > Hello!
> >
> > As you have have noticed the download counts on PyPI are no longer
> > updating. Originally this was due to an issue with the script that
> > processes these download counts. However I have now removed the download
> > counts from the PyPI webui and their use via the API is considered
> > deprecated.
> >
> > There are numerous reasons for their removal/deprecation some of which are:
> >    - Technically hard to make work with the new CDN
> >        - The CDN is being donated to the PSF, and the donated tier does
> > not offer any form of log access
> >
> >
> > What would be involved money/effort wise to get such access?
> >
> >
> > I didn't see an answer to this.
> >
> > Any idea how much this would cost?  With access to logs, we could
> > compute download counts.
> >
> > Jim
> >
> > --
> > Jim Fulton
> > http://www.linkedin.com/in/jimfulton
> >
> >
> > Fastly has given us access to their streaming log support.
> >
> 
> That's another deal. This is just what is expected in exchange for the
> "Real-time CDN by Fastly" phrase at the bottom of the page to make it
> reflect the reality.
> 
> I'd say that not getting access to download stats from the start is a fail
> on the side of PSF or whoever involved in coordination with Fastly. 

I also was surprised when download counts were gone but download stats
aren't very reliable indicators, anyway.  To begin with, one might configure
a CI system to cache pypi packages, another might re-download all the time.
Some popular packages are rather used from system distros, others downloaded
from pypi.  And there are many more issues.  Seems like there are not 
overwhelmingly many people unhappy about its removal.  Doesn't mean
they shouldn't be reinstanted, just saying.

> There
> are many CDN providers out there and I suspect so far only Fastly was
> contacted. The primary responsibility of that coordinator is correctly
> sending the message for CDN provider that PyPI is a public exhibition of
> their service quality, and not a tax exemption for charity. I'd say Fastly
> should be interested to help with making us download stats exposed in a
> convenient API friendly way, because real-time stats is the key feature of
> their marketing advantage as I see it.

FYI Fastly has been responding swiftly and responsibly so far, as far as
i could follow it.  And Donald has been helping people on various fronts
(not only Fastly related) and been the main driver of the CDN move,
thankfully.  Things are starting to work more smoothly now and i guess
some bumps in the road were unavoidable because they only show up in
real life.

cheers,
holger


> Infrastructure will be setting up a secure method for receiving these logs
> > at which point PyPI can use them.
> >
> 
> Is it done already? I am free to some degree to help with that. What is the
> current problem to tackle on?

> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig