<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><br><div><div>On Oct 25, 2013, at 9:35 PM, Nick Coghlan <<a href="mailto:ncoghlan@gmail.com">ncoghlan@gmail.com</a>> wrote:</div><blockquote type="cite"><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br><p dir="ltr">Most downloads happen through the Fastly CDN - the numbers are derived from the Fastly logs rather than being direct. The code that does that log analysis is in<span class="Apple-converted-space"> </span><a href="https://bitbucket.org/pypa/pypi/src">https://bitbucket.org/pypa/pypi/src</a><span class="Apple-converted-space"> </span>(Donald would be able to provide a more direct reference to the relevant source).</p></div></blockquote><div><br></div><div><a href="https://bitbucket.org/pypa/pypi/src/0c749f947b167b6643ed94ceac2b3e1ab478d5bd/tools/rsyslog-cdn.py?at=default">https://bitbucket.org/pypa/pypi/src/0c749f947b167b6643ed94ceac2b3e1ab478d5bd/tools/rsyslog-cdn.py?at=default</a></div><div><br></div><div>It buckets data and puts it into redis. Really “dumb” but it works well enough until better infrastructure can be put into place.</div><blockquote type="cite"><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><p dir="ltr">However, separating downloads between mirroring, automatic deployments and integration and actual direct downloads isn't something PyPI has ever done, or is really able to do in a systematic way. "pip install thatproject" (and equivalent commands for other tools) looks the same to PyPI regardless of whether it's a human or a script running the command.</p><p dir="ltr">That's why Donald's recent download analysis was able to split it up by tools, but not by purpose.</p></div></blockquote>Yea this part is hard/impossible :/<br><blockquote type="cite"><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><p dir="ltr">Now, exposing more of that analytical data to package owners on an ongoing basis is an interesting idea, but one that would be a *very* long way down the priority list for the current development team.</p></div></blockquote><div><br></div>Nice analytics for package owners is on the road map, but it’s, as you mentioned, down the road map a ways.<br><blockquote type="cite"><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><p dir="ltr">However, if someone else were to figure out a way to expose the data users needed to do their own analysis, it might be possible to support that, although it may be better to look at offering that through Warehouse (aka PyPI.next) rather than the existing PyPI software (<a href="https://github.com/dstufft/warehouse">https://github.com/dstufft/warehouse</a>). There's a demo instance (using live data) running at<span class="Apple-converted-space"> </span><a href="http://preview-pypi.python.org/">preview-pypi.python.org</a>, but that's mostly focused on backwards compatibility testing for the tool APIs at this point rather than being navigable through a web browser.</p></div></blockquote></div><div apple-content-edited="true">-----------------<br>Donald Stufft<br>PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
</div>
<br></body></html>