Re: [devpi-dev] changing pypi polling/changelog logic
On Wed, Jun 03, 2015 at 16:48 +0200, Florian Schulze wrote:
So I propose roughly the following:
...
- if a mirror page is accessed and has recent enough information (say 30 minutes by default) use the cached info, otherwise refetch/refresh and update cached information if there are changes. If there is old information and the old site can not be reached, simply use the old information, emit a warning in the log.
If the background thread is running, the refresh should be triggered in the background as well. That way if I don't have internet, I don't have to wait for the timeout all the time I'm installing something.
Do you mean that if a simple page is more than 30 minutes old we would start a background refresh task but otherwise serve the cached information? If you are offline then trying to get from pypi.python.org should immediately fail and not introduce a significant hold up for serving the cached information (to be verified of course). So i'd hope to be able to avoid any background thread. If in doubt we could increase the default timeout and/or set it to "0" meaning you have to manually refresh always. holger
Otherwise I can't think of anything speaking against your points.
Regards, Florian Schulze
-- about me: http://holgerkrekel.net/about-me/ contracting: http://merlinux.eu
participants (1)
-
holger krekel