[Patches] protection for marshalling recursive data structures

Michael Hudson mwh21@cam.ac.uk
Mon, 19 Jun 2000 01:20:50 +0100 (BST)


As I really do not have anything better to do at the moment, I've written
a patch to Python/marshal.c that prevents Python dumping core when trying
to marshal stack bustingly deep (or recursive) data structure.

It just throws an exception; even slightly clever handling of recursive
data is what pickle is for...

Index: marshal.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/marshal.c,v
retrieving revision 1.47
diff -u -r1.47 marshal.c
--- marshal.c	2000/05/03 23:44:39	1.47
+++ marshal.c	2000/06/19 00:18:44
@@ -58,6 +58,7 @@
 typedef struct {
 	FILE *fp;
 	int error;
+	int depth;
 	/* If fp == NULL, the following are valid: */
 	PyObject *str;
 	char *ptr;
@@ -144,8 +145,13 @@
 {
 	int i, n;
 	PyBufferProcs *pb;
+
+	p->depth++;
 	
-	if (v == NULL) {
+	if (p->depth > 5000) {
+		p->error = 2;
+	} 
+	else if (v == NULL) {
 		w_byte(TYPE_NULL, p);
 	}
 	else if (v == Py_None) {
@@ -301,6 +307,7 @@
 	WFILE wf;
 	wf.fp = fp;
 	wf.error = 0;
+	wf.depth = 0;
 	w_long(x, &wf);
 }
 
@@ -690,6 +697,7 @@
 	wf.ptr = PyString_AS_STRING((PyStringObject *)wf.str);
 	wf.end = wf.ptr + PyString_Size(wf.str);
 	wf.error = 0;
+	wf.depth = 0;
 	w_object(x, &wf);
 	if (wf.str != NULL)
 		_PyString_Resize(&wf.str,
@@ -697,7 +705,9 @@
 			   PyString_AS_STRING((PyStringObject *)wf.str)));
 	if (wf.error) {
 		Py_XDECREF(wf.str);
-		PyErr_SetString(PyExc_ValueError, "unmarshallable object");
+		PyErr_SetString(PyExc_ValueError, 
+				(wf.error==1)?"unmarshallable object"
+				:"object too deeply nested to marshal");
 		return NULL;
 	}
 	return wf.str;
@@ -724,9 +734,12 @@
 	wf.str = NULL;
 	wf.ptr = wf.end = NULL;
 	wf.error = 0;
+	wf.depth = 0;
 	w_object(x, &wf);
 	if (wf.error) {
-		PyErr_SetString(PyExc_ValueError, "unmarshallable object");
+		PyErr_SetString(PyExc_ValueError, 
+				(wf.error==1)?"unmarshallable object"
+				:"object too deeply nested to marshal");
 		return NULL;
 	}
 	Py_INCREF(Py_None);

On my machine, it seems an unpatched marshal blows up at a depth of about
13000, so I hope 5000 is OK almost everywhere.

I confirm that, to the best of my knowledge and belief, this
contribution is free of any claims of third parties under
copyright, patent or other rights or interests ("claims").  To
the extent that I have any such claims, I hereby grant to CNRI a
nonexclusive, irrevocable, royalty-free, worldwide license to
reproduce, distribute, perform and/or display publicly, prepare
derivative versions, and otherwise use this contribution as part
of the Python software and its related documentation, or any
derivative versions thereof, at no cost to CNRI or its licensed
users, and to authorize others to do so.

I acknowledge that CNRI may, at its sole discretion, decide
whether or not to incorporate this contribution in the Python
software and its related documentation.  I further grant CNRI
permission to use my name and other identifying information
provided to CNRI by me for use in connection with the Python
software and its related documentation.

Cheers,
M.