[Python-Dev] buglet in long("123\0", 10)

[Calvin Spealman]

Added a check in test_long.LongTest.test_misc() that long("123\0", 10)
fails properly and adapted the patch to int_new to long_new. I get
this weird feeling that if its impossible for the function
(PyLong_FromString) to know if its being given bad data, having know
way to know if the string is supposed to continue past the zero-byte,
then doesn't it make sense to say that the function by design is
broken? Anyway, here is the patch.

Index: Objects/longobject.c
===================================================================
--- Objects/longobject.c        (revision 53406)
+++ Objects/longobject.c        (working copy)
@@ -3287,10 +3287,27 @@
                return PyLong_FromLong(0L);
        if (base == -909)
                return PyNumber_Long(x);
-       else if (PyString_Check(x))
+       if (PyString_Check(x)) {
+               /* Since PyLong_FromString doesn't have a length parameter,
+                * check here for possible NULs in the string. */
+               char *string = PyString_AS_STRING(x);
+               if (strlen(string) != PyString_Size(x)) {
+                       /* create a repr() of the input string,
+                        * just like PyLong_FromString does. */
+                       PyObject *srepr;
+                       srepr = PyObject_Repr(x);
+                       if (srepr == NULL)
+                               return NULL;
+                       PyErr_Format(PyExc_ValueError,
+                            "Invalid literal for long() with base %d: %s",
+                            base, PyString_AS_STRING(srepr));
+                       Py_DECREF(srepr);
+                       return NULL;
+               }
                return PyLong_FromString(PyString_AS_STRING(x), NULL, base);
+       }
 #ifdef Py_USING_UNICODE
-       else if (PyUnicode_Check(x))
+       if (PyUnicode_Check(x))
                return PyLong_FromUnicode(PyUnicode_AS_UNICODE(x),
                                          PyUnicode_GET_SIZE(x),
                                          base);
@@ -3439,3 +3456,4 @@
        long_new,                               /* tp_new */
        PyObject_Del,                           /* tp_free */
 };
+
Index: Lib/test/test_long.py
===================================================================
--- Lib/test/test_long.py       (revision 53406)
+++ Lib/test/test_long.py       (working copy)
@@ -299,6 +299,8 @@
         slicemin, slicemax = X()[-2L**100:2L**100]
         self.assertEqual(X()[slicemin:slicemax], (slicemin, slicemax))

+        self.assertRaises(ValueError, long, "123\0", 10)
+
 # ----------------------------------- tests of auto int->long conversion

     def test_auto_overflow(self):


### END PATCH

On 1/15/07, Neal Norwitz <nnorwitz at gmail.com> wrote:
> Calvin,
>
> Was Guido's answer enough or did you want some more?  If you take a
> look at the svn rev I sent, I think it should be pretty easy to come
> up with a patch for long too.
>
> Getting the size won't help (exactly), since the problem is the diff
> between strlen() and the size.  You can see this in the int() fix.
>
> Let me know if you have any questions.  It would be great if you could
> produce a patch.
>
> Cheers,
> n
> --
> On 1/14/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> > Is it a more general problem that null-terminated strings are used
> > with data from strings we specifically allow to contain null bytes?
> > Perhaps a migration of *FromString() to *FromStringAndSize()
> > functions, or taking Python string object pointers, would be a more
> > general solution to set as a goal, so this sort of thing can't crop up
> > down the road, again.
> >
> > I know I'm still very uninitiated in the internals of CPython, so
> > anyone please correct me if my thoughts here are against any on-going
> > policy or reasoning.
> >
> > On 1/14/07, Neal Norwitz <nnorwitz at gmail.com> wrote:
> > > SVN rev 52305 resolved Bug #1545497: when given an explicit base,
> > > int() did ignore NULs embedded in the string to convert.
> > >
> > > However, the same fix wasn't applied for long().
> > >
> > > n
> > >
> > > On 1/13/07, Guido van Rossum <guido at python.org> wrote:
> > > > What's wrong with this session? :-)
> > > >
> > > > Python 2.6a0 (trunk:53416, Jan 13 2007, 15:24:17)
> > > > [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
> > > > Type "help", "copyright", "credits" or "license" for more information.
> > > > >>> int('123\0')
> > > > Traceback (most recent call last):
> > > >   File "<stdin>", line 1, in <module>
> > > > ValueError: null byte in argument for int()
> > > > >>> int('123\0', 10)
> > > > Traceback (most recent call last):
> > > >   File "<stdin>", line 1, in <module>
> > > > ValueError: invalid literal for int() with base 10: '123\x00'
> > > > >>> long('123\0')
> > > > Traceback (most recent call last):
> > > >   File "<stdin>", line 1, in <module>
> > > > ValueError: null byte in argument for long()
> > > > >>> long('123\0', 10)
> > > > 123L
> > > > >>>
> > > >
> > > > --
> > > > --Guido van Rossum (home page: http://www.python.org/~guido/)
> > > > _______________________________________________
> > > > Python-Dev mailing list
> > > > Python-Dev at python.org
> > > > http://mail.python.org/mailman/listinfo/python-dev
> > > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
> > > >
> > > _______________________________________________
> > > Python-Dev mailing list
> > > Python-Dev at python.org
> > > http://mail.python.org/mailman/listinfo/python-dev
> > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com
> > >
> >
> >
> > --
> > Read my blog! I depend on your acceptance of my opinion! I am interesting!
> > http://ironfroggy-code.blogspot.com/
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com
> >
>


-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/