[Python-ideas] Cross-platform pickling of Path objects

[Nick Coghlan]

On 16 July 2016 at 15:51, Antony Lee <antony.lee at berkeley.edu> wrote:
> Sure.  I use pickles to store "processed" (scientific) datasets; they
> typically keep the path to a "raw" dataset as an object attribute (of a Path
> class, either PosixPath or WindowsPath depending on the "OS" (POSIX/Windows)
> of the computer where the processing is originally done), as it is sometimes
> useful to go back and check the raw datasets.  However, in general the
> processed datasets are enough by themselves, so it makes sense to transfer
> them (independently of the raw datasets) to another computer before further
> processing.
>
> Obviously, once the processed dataset has been transferred to another
> computer, trying to access the raw dataset will fail with an OSError (e.g.,
> FileNotFoundError); I have code in place to handle that.  However, if the
> file is transferred from a POSIX "OS" to a Windows "OS" or vice-versa, it
> becomes impossible to even unpickle the file.
>
> Converting Path objects to PurePaths upon unpickling on a different "OS",
> either silently or possibly with a warning, would allow me to just catch
> both OSErrors and AttributeErrors when trying to access the raw dataset from
> the path attribute of the processed one.

The approach of converting to a PurePath with a RuntimeWarning when
unpickling seems reasonable to me - if a particular project wants to
silence that warning, they can use the warning machinery to suppress
it when loading the pickled data.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia