Hi, I'm hoping you can advise me. The sqlite database has become corrupted due to a disk problem. It was not on the same drive as the raw files so they are all in tact. I'm assuming there is a way in which I can restore the files back but also this may be rather involved. Is this possible? We will have pushed local packages to local indexes over about a week or two and I'd like to recover those. Thanks in advance Dominic
On 5 Jul 2021, at 19:30, dfjbinks@gmail.com wrote:
Hi,
Hi!
I'm hoping you can advise me. The sqlite database has become corrupted due to a disk problem. It was not on the same drive as the raw files so they are all in tact.
Is the database completely gone? There may be ways to recover at least partial data from it. Was there a specific reason to split the data? How did you do the split?
I'm assuming there is a way in which I can restore the files back but also this may be rather involved.
Is this possible? We will have pushed local packages to local indexes over about a week or two and I'd like to recover those.
The files are stored per user/index + the hash and the regular filename, so you can rather easily get to them. Most metadata should also be recoverable by re-uploading them to a new installation. Some metadata might be lost though. You should go through the directory structure to get a feel for the layout and see if you find what you need. If you have further questions, don't hesitate to ask. Regards, Florian Schulze
Thanks for the swift response. On 06/07/2021 07:02, Florian Schulze wrote:
On 5 Jul 2021, at 19:30, dfjbinks@gmail.com wrote:
Hi,
Hi!
I'm hoping you can advise me. The sqlite database has become corrupted due to a disk problem. It was not on the same drive as the raw files so they are all in tact.
Is the database completely gone? There may be ways to recover at least partial data from it.
Was there a specific reason to split the data? How did you do the split?
Yes - it's completely gone. I did something stupid. The devpi installation is on an NFS filesystem. Since the database, I assumed, was accessed frequently and therefore could be causing a performance issue, I moved the database to a local filesystem and placed a symlink in the devpi directory. I put the database in /var. Only it was a part of /var that was a temporary filesystem. A subsequent kernel panic led to the loss of the database. And yes. I am an idiot.
I'm assuming there is a way in which I can restore the files back but also this may be rather involved.
Is this possible? We will have pushed local packages to local indexes over about a week or two and I'd like to recover those.
The files are stored per user/index + the hash and the regular filename, so you can rather easily get to them. Most metadata should also be recoverable by re-uploading them to a new installation. Some metadata might be lost though.
You should go through the directory structure to get a feel for the layout and see if you find what you need. If you have further questions, don't hesitate to ask.
I actually did this yesterday and found the relevant files. I have found 68 files that are more recent than the relocation of the database and since the files are paired because they are docs + python package - there are 34 such uploads. I will most likely reupload the packages one at a time since the number is small. The loss of the metadata is unlikely to be a problem since these are uploads from a CI system and probably won't even be used.
Regards, Florian Schulze
On 6 Jul 2021, at 9:37, Dominic Binks wrote:
Thanks for the swift response.
On 06/07/2021 07:02, Florian Schulze wrote:
On 5 Jul 2021, at 19:30, dfjbinks@gmail.com wrote:
Hi,
Hi!
I'm hoping you can advise me. The sqlite database has become corrupted due to a disk problem. It was not on the same drive as the raw files so they are all in tact.
Is the database completely gone? There may be ways to recover at least partial data from it.
Was there a specific reason to split the data? How did you do the split?
Yes - it's completely gone. I did something stupid. The devpi installation is on an NFS filesystem. Since the database, I assumed, was accessed frequently and therefore could be causing a performance issue, I moved the database to a local filesystem and placed a symlink in the devpi directory. I put the database in /var. Only it was a part of /var that was a temporary filesystem. A subsequent kernel panic led to the loss of the database. And yes. I am an idiot.
Using NFS is discouraged, due to the way locking works. It kinda works for files, but definitely not for the database. Also if you use the search from devpi-web, the used library whoosh most likely doesn't like NFS either.
I'm assuming there is a way in which I can restore the files back but also this may be rather involved.
Is this possible? We will have pushed local packages to local indexes over about a week or two and I'd like to recover those.
The files are stored per user/index + the hash and the regular filename, so you can rather easily get to them. Most metadata should also be recoverable by re-uploading them to a new installation. Some metadata might be lost though.
You should go through the directory structure to get a feel for the layout and see if you find what you need. If you have further questions, don't hesitate to ask.
I actually did this yesterday and found the relevant files. I have found 68 files that are more recent than the relocation of the database and since the files are paired because they are docs + python package - there are 34 such uploads.
I will most likely reupload the packages one at a time since the number is small. The loss of the metadata is unlikely to be a problem since these are uploads from a CI system and probably won't even be used.
That sounds like the simplest solution for the severity level. Regards, Florian Schulze
How to download Play Store for different devices? https://filetodown.com/ https://www.googleplay-apk.com https://www.softaty.com/android/google-play/
participants (6)
-
dfjbinks＠gmail.com -
Dominic Binks -
Elenajames546＠protonmail.com -
Florian Schulze -
pamelaconley333＠yahoo.com -
playstorefile＠gmail.com