[Python-Dev] Newly Built Python3 Binary Throws Segfault

[Cyd Haselton]

Question:
When you said earlier that you found the problem by using grep...were you looking for places where strdup called locale?

On January 30, 2015 7:52:47 PM CST, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>Regardless, if you're looking to toy more with stuff like this, I'd
>highly
>recommend dual-booting with Ubuntu, which is what I'm doing now. (Now I
>rarely ever boot into Windows!)
>
>On Fri, Jan 30, 2015 at 7:51 PM, Ryan Gonzalez <rymg19 at gmail.com>
>wrote:
>
>> Do you have just the SDK (which doesn't require Cygwin) or a rooted
>phone?
>> If so, I can upload instructions that don't use the NDK.
>>
>> On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com>
>wrote:
>>
>>> This is going to take some time...here's why:
>>>
>>> Downloading and installing the NDK/SDK won't be too hard...I have to
>>> clear some space...but my primary machine is running Windows 7 and
>I'd
>>> rather swallow hot coals than install Cygwin.  I've got next to no
>>> experience with it, other than knowing that the NDK recommends
>against
>>> it.
>>>
>>> I've got a CentOS VM...but it's currently in tarball form on an
>>> external drive for space reasons...and it only has the NDK
>installed.
>>> If I am able to get it back up and running, there's still the task
>of
>>> getting  adb connected to my device. from the VM.
>>>
>>> Not saying it's impossible...just that it'll take time...and I'll
>>> probably have to tackle it tomorrow (earliest) or Sunday (latest). 
>In
>>> the meantime I'll also check to see if there's anything that can a)
>>> run in an Android terminal and b) can take a stack trace; it would
>be
>>> far, far, far easier than either option above.
>>>
>>>
>>>
>>> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com>
>wrote:
>>> > Do you have the Android SDK and NDK installed? If so...
>>> >
>>> > Using Google, I've created this series of steps, which may (or may
>not)
>>> > work:
>>> >
>>> > 1. Make sure you have a copy of Python on your computer and make
>sure
>>> that
>>> > it's built with debug symbols.
>>> >
>>> > 2. Run the following commands from a shell with your phone plugged
>in:
>>> >
>>> >   adb forward tcp:5039 tcp:5039
>>> >   adb shell /system/bin/gdbserver tcp:5039 <path to python
>executable>
>>> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
>>> >
>>> > Now, GDB should have opened, so type the following commands:
>>> >
>>> >   set solib-search-path <directory holder libpython.so>
>>> >   file <path to local python executable>
>>> >   target remote :5055
>>> >   run
>>> >
>>> > Now, wait for the program to crash and type:
>>> >
>>> >   backtrace
>>> >
>>> > You should now see a complete backtrace in your terminal. To GDB,
>type:
>>> >
>>> >   quit
>>> >
>>> > *crosses fingers*
>>> >
>>> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton
><chaselton at gmail.com>
>>> wrote:
>>> >>
>>> >> Unfortunately it is still reporting the same function :-/.
>>> >>
>>> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>> wrote:
>>> >> > Yes...
>>> >> >
>>> >> > Can you check if it's crashing in a different function now?
>>> >> >
>>> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton
><chaselton at gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> Yes I did.  I did have to enter all the information in
>>> manually...I'm
>>> >> >> not familiar with automated patch application tools and even
>if I
>>> >> >> were, I'm pretty sure I wouldn't have them on my device.
>>> >> >>
>>> >> >> Just so that I'm absolutely sure I got everything
>correct...you
>>> wanted
>>> >> >> all of the lines in the patch commented out, correct? 
>Basically
>>> >> >> everything referencing oldloc?
>>> >> >>
>>> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez
><rymg19 at gmail.com>
>>> >> >> wrote:
>>> >> >> > Are you sure the patch was applied correctly? I was SO sure
>it
>>> would
>>> >> >> > work!
>>> >> >> >
>>> >> >> > FYI, you tried the patch I attached to the email message,
>right?
>>> >> >> >
>>> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
>>> chaselton at gmail.com>
>>> >> >> > wrote:
>>> >> >> >>
>>> >> >> >> Update:  I did try the patch after getting it installed
>>> correctly,
>>> >> >> >> but
>>> >> >> >> I'm still getting a segfault on the newly built binary.
>>> >> >> >> Will post info this afternoon.
>>> >> >> >>
>>> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <
>>> rymg19 at gmail.com>
>>> >> >> >> wrote:
>>> >> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>>> >> >> >> > unconditionally
>>> >> >> >> > checks the length of the (possibly null) string argument
>first.
>>> >> >> >> >
>>> >> >> >> > Please try the patch I attached in the last email. It
>*might*
>>> fix
>>> >> >> >> > the
>>> >> >> >> > issue.
>>> >> >> >> > Android has crappy locale handling.
>>> >> >> >> >
>>> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>>> >> >> >> > <chaselton at gmail.com>
>>> >> >> >> > wrote:
>>> >> >> >> >>
>>> >> >> >> >>   Unless i'm reading something incorrectly, 
>_PyMem_RawStrdup
>>> is
>>> >> >> >> >> currently returning NULL when given a null pointer.
>>> >> >> >> >>
>>> >> >> >> >> From obmalloc.c
>>> >> >> >> >>
>>> >> >> >> >> _PyMem_RawStrdup(const char *str)
>>> >> >> >> >> {
>>> >> >> >> >>     size_t size;
>>> >> >> >> >> char *copy;
>>> >> >> >> >>     size = strlen(str) + 1;
>>> >> >> >> >>     copy = PyMem_RawMalloc(size);
>>> >> >> >> >>     if (copy == NULL)
>>> >> >> >> >>         return NULL;
>>> >> >> >> >> memcpy(copy, str, size);
>>> >> >> >> >> return copy;
>>> >> >> >> >>           }
>>> >> >> >> >>
>>> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>>> >> >> >> >> <rymg19 at gmail.com>
>>> >> >> >> >> wrote:
>>> >> >> >> >> > I seriously doubt the issue is in that file;
>>> _PyMem_RawStrdup
>>> >> >> >> >> > crashes
>>> >> >> >> >> > when
>>> >> >> >> >> > calling strlen. It's that whatever is calling it is
>likely
>>> >> >> >> >> > asking
>>> >> >> >> >> > it
>>> >> >> >> >> > to
>>> >> >> >> >> > duplicate a null pointer. Basically, it's probably the
>>> caller's
>>> >> >> >> >> > fault.
>>> >> >> >> >> >
>>> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to
>return
>>> NULL
>>> >> >> >> >> > when
>>> >> >> >> >> > given a
>>> >> >> >> >> > null pointer and see where it then segfaults.
>>> >> >> >> >> >
>>> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>>> >> >> >> >> > <chaselton at gmail.com>
>>> >> >> >> >> > wrote:
>>> >> >> >> >> >>
>>> >> >> >> >> >> Alternatively, is there a hassle-free way to find out
>what
>>> >> >> >> >> >> changed
>>> >> >> >> >> >> in
>>> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>>> >> >> >> >> >>
>>> >> >> >> >> >>
>>> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>>> >> >> >> >> >> <chaselton at gmail.com>
>>> >> >> >> >> >> wrote:
>>> >> >> >> >> >> > There's a related strdup patch for readline.c,
>mentioned
>>> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>>> >> >> >> >> >> >
>https://github.com/rave-engine/python3-android/issues/2.
>>> >> >> >> >> >> > There's a patch, but I'm not sure how to modify it
>for
>>> >> >> >> >> >> > obmalloc.c,
>>> >> >> >> >> >> > as
>>> >> >> >> >> >> > (I think) the functions all belong to
>Python...they're
>>> all
>>> >> >> >> >> >> > prefixed
>>> >> >> >> >> >> > with _PyXx
>>> >> >> >> >> >> >
>>> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>>> >> >> >> >> >> > <chaselton at gmail.com>
>>> >> >> >> >> >> > wrote:
>>> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on device
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>>> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>>> >> >> >> >> >> >> 0008bbc8
>>> >> >> >> >> >> >> _PyMem_RawStrdup
>>> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan
><rymg19 at gmail.com
>>> >
>>> >> >> >> >> >> >> wrote:
>>> >> >> >> >> >> >>> Could you try the steps at
>>> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>>> >> >> >> >> >> >>> allow you to get a better idea of where libc is
>>> crashing.
>>> >> >> >> >> >> >>>
>>> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> Managed to get this out of logcat:
>>> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000
>>> >> >> >> >> >> >>>> (code=1),
>>> >> >> >> >> >> >>>> thread
>>> >> >> >> >> >> >>>> 11914 (python)  (libc)
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000
>(code=1),
>>> thread
>>> >> >> >> >> >> >>>> 23373
>>> >> >> >> >> >> >>>> (python)
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> Less detail than strace but it seems to be that
>>> python is
>>> >> >> >> >> >> >>>> segfaulting
>>> >> >> >> >> >> >>>> libc...
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>>> >> >> >> >> >> >>>> <rymg19 at gmail.com>
>>> >> >> >> >> >> >>>> wrote:
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van
>Rossum
>>> >> >> >> >> >> >>>>> <guido at python.org>
>>> >> >> >> >> >> >>>>> wrote:
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  What I see in the strace:
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>>> >> >> >> >> >> >>>>>>  ... load libm
>>> >> >> >> >> >> >>>>>>  ... open /dev/__properties__ and do something
>to it
>>> >> >> >> >> >> >>>>>> (what?)
>>> >> >> >> >> >> >>>>>>  ... get current time
>>> >> >> >> >> >> >>>>>>  ... allocate memory
>>> >> >> >> >> >> >>>>>>  ... getuid
>>> >> >> >> >> >> >>>>>>  ... segfault
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't
>look as
>>> if
>>> >> >> >> >> >> >>>>>> it
>>> >> >> >> >> >> >>>>>> has
>>> >> >> >> >> >> >>>>>> started to
>>> >> >> >> >> >> >>>>>>  load modules yet.
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to
>me.
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>>
>https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150131/a38b1b30/attachment-0001.html>