[Numpy-discussion] Question about dtype

Valentin Haenel valentin at haenel.co
Fri Dec 26 20:23:59 EST 2014


* Valentin Haenel <valentin at haenel.co> [2014-12-27]:
> * Nathaniel Smith <njs at pobox.com> [2014-12-13]:
> [snip]
> > Ah, so your question is about how to serialize dtypes.
> > 
> > The simplest approach would be to use pickle and shove the resulting string
> > into your json. However, this is very dangerous if you need to process
> > untrusted files, because if I can convince you to unpickle an arbitrary
> > string, then I can run arbitrary code on your computer.
> > 
> > I believe .npy file format has a safe method for (un)serializing drypes.
> > I'd look up what it does.
> 
> Just to follow this up:
> 
> NPY actually does some magic to differntiate between simple and complex
> dtypes (I had already discovered this and am doing it too):
> 
> https://github.com/numpy/numpy/blob/master/numpy/lib/format.py#L210
> 
> And then it does a ``repr`` on the result:
> 
> https://github.com/numpy/numpy/blob/master/numpy/lib/format.py#L290
> 
> On loading it does a ``(safe_)eval`` on the whole header dict:
> 
> https://github.com/numpy/numpy/blob/master/numpy/lib/format.py#L479
> 
> I can do that too and it is what was suggested later on in this thread,
> but simple dtypes cause a SyntaxError. So what I'll do is try to
> safe_eval the string, catch the SyntaxError and just use the plain
> string in that case. That should be easier than trying to reassmble the
> correct thing from the deserialzed JSON.

Sorry, my bad, if I repr a string, of course I can eval it. ;)

V-



More information about the NumPy-Discussion mailing list