Hello, I have a question about maintenance of devpi. It seems like if used longer term, it has a couple of point that need to be addressed one way or another: 1. Constantly growing number of cached pypi files. 2. Constantly growing size of sqlite database. Are there any ways to manage amount of disk space required in both cases? How do people usually deal with this? By increasing disk space, maybe fully redeploying devpi? Kind Regards, Sarunas -- Mintel Group Ltd | 11 Pilgrim Street | London | EC4V 6RN Registered in England: Number 1475918. | VAT Number: GB 232 9342 72 Contact details for our other offices can be found at http://www.mintel.com/office-locations. This email and any attachments may include content that is confidential, privileged or otherwise protected under applicable law. Unauthorised disclosure, copying, distribution or use of the contents is prohibited and may be unlawful. If you have received this email in error, including without appropriate authorisation, then please reply to the sender about the error and delete this email and any attachments.
On 1 Aug 2017, at 23:30, Šarūnas Saulis wrote:
Hello,
I have a question about maintenance of devpi. It seems like if used longer term, it has a couple of point that need to be addressed one way or another:
1. Constantly growing number of cached pypi files.
The files are in [server storage directory]/+files/root/pypi/+f and can be safely deleted. Devpi will re-download them on demand. Just be careful to not touch any other files. It's best to move them away first, so you can restore them in case of issues. We have plans to add a prune command which allows to remove things based on various filters (age, version, release type etc).
2. Constantly growing size of sqlite database.
The only way to reduce the sqlite database size is an export/import cycle. I'd be interested to know what amount of sqlite db growth you see. With current devpi this shouldn't be much of an issue anymore, as we have moved most mirror info out of the db. The biggest source of growth is when you upload a lot of packages via a CI system for example. Regards, Florian Schulze
I think it might be about a year that our current set up is running, and we currently have ~10G sqlite database. Obivously it's not a massive problem from disk space point of view, but it would be nice to have ability to control its growth. We do build and make our own packages available on devpi for each commit, so that would explain why sqlite is so big. We are however running an older version of devpi, and will be upgrading to the latest version at some point. Kind Regards, Sarunas Saulis
On 1 Aug 2017, at 23:30, Šarūnas Saulis wrote:
The files are in [server storage directory]/+files/root/pypi/+f and can be safely deleted. Devpi will re-download them on demand. Just be careful to not touch any other files. It's best to move them away first, so you can restore them in case of issues.
We have plans to add a prune command which allows to remove things based on various filters (age, version, release type etc).
The only way to reduce the sqlite database size is an export/import cycle.
I'd be interested to know what amount of sqlite db growth you see. With current devpi this shouldn't be much of an issue anymore, as we have moved most mirror info out of the db. The biggest source of growth is when you upload a lot of packages via a CI system for example.
Regards, Florian Schulze
participants (2)
-
Florian Schulze -
Šarūnas Saulis