[Python-Dev] Newly Built Python3 Binary Throws Segfault
Cyd Haselton
chaselton at gmail.com
Sat Jan 31 01:19:56 CET 2015
This is going to take some time...here's why:
Downloading and installing the NDK/SDK won't be too hard...I have to
clear some space...but my primary machine is running Windows 7 and I'd
rather swallow hot coals than install Cygwin. I've got next to no
experience with it, other than knowing that the NDK recommends against
it.
I've got a CentOS VM...but it's currently in tarball form on an
external drive for space reasons...and it only has the NDK installed.
If I am able to get it back up and running, there's still the task of
getting adb connected to my device. from the VM.
Not saying it's impossible...just that it'll take time...and I'll
probably have to tackle it tomorrow (earliest) or Sunday (latest). In
the meantime I'll also check to see if there's anything that can a)
run in an Android terminal and b) can take a stack trace; it would be
far, far, far easier than either option above.
On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> Do you have the Android SDK and NDK installed? If so...
>
> Using Google, I've created this series of steps, which may (or may not)
> work:
>
> 1. Make sure you have a copy of Python on your computer and make sure that
> it's built with debug symbols.
>
> 2. Run the following commands from a shell with your phone plugged in:
>
> adb forward tcp:5039 tcp:5039
> adb shell /system/bin/gdbserver tcp:5039 <path to python executable>
> <path to ndk binary folder>/arm-linux-androideabi-gdb
>
> Now, GDB should have opened, so type the following commands:
>
> set solib-search-path <directory holder libpython.so>
> file <path to local python executable>
> target remote :5055
> run
>
> Now, wait for the program to crash and type:
>
> backtrace
>
> You should now see a complete backtrace in your terminal. To GDB, type:
>
> quit
>
> *crosses fingers*
>
> On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>>
>> Unfortunately it is still reporting the same function :-/.
>>
>> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>> > Yes...
>> >
>> > Can you check if it's crashing in a different function now?
>> >
>> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton <chaselton at gmail.com>
>> > wrote:
>> >>
>> >> Yes I did. I did have to enter all the information in manually...I'm
>> >> not familiar with automated patch application tools and even if I
>> >> were, I'm pretty sure I wouldn't have them on my device.
>> >>
>> >> Just so that I'm absolutely sure I got everything correct...you wanted
>> >> all of the lines in the patch commented out, correct? Basically
>> >> everything referencing oldloc?
>> >>
>> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> wrote:
>> >> > Are you sure the patch was applied correctly? I was SO sure it would
>> >> > work!
>> >> >
>> >> > FYI, you tried the patch I attached to the email message, right?
>> >> >
>> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <chaselton at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Update: I did try the patch after getting it installed correctly,
>> >> >> but
>> >> >> I'm still getting a segfault on the newly built binary.
>> >> >> Will post info this afternoon.
>> >> >>
>> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> >> wrote:
>> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>> >> >> > unconditionally
>> >> >> > checks the length of the (possibly null) string argument first.
>> >> >> >
>> >> >> > Please try the patch I attached in the last email. It *might* fix
>> >> >> > the
>> >> >> > issue.
>> >> >> > Android has crappy locale handling.
>> >> >> >
>> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>> >> >> > <chaselton at gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Unless i'm reading something incorrectly, _PyMem_RawStrdup is
>> >> >> >> currently returning NULL when given a null pointer.
>> >> >> >>
>> >> >> >> From obmalloc.c
>> >> >> >>
>> >> >> >> _PyMem_RawStrdup(const char *str)
>> >> >> >> {
>> >> >> >> size_t size;
>> >> >> >> char *copy;
>> >> >> >> size = strlen(str) + 1;
>> >> >> >> copy = PyMem_RawMalloc(size);
>> >> >> >> if (copy == NULL)
>> >> >> >> return NULL;
>> >> >> >> memcpy(copy, str, size);
>> >> >> >> return copy;
>> >> >> >> }
>> >> >> >>
>> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>> >> >> >> <rymg19 at gmail.com>
>> >> >> >> wrote:
>> >> >> >> > I seriously doubt the issue is in that file; _PyMem_RawStrdup
>> >> >> >> > crashes
>> >> >> >> > when
>> >> >> >> > calling strlen. It's that whatever is calling it is likely
>> >> >> >> > asking
>> >> >> >> > it
>> >> >> >> > to
>> >> >> >> > duplicate a null pointer. Basically, it's probably the caller's
>> >> >> >> > fault.
>> >> >> >> >
>> >> >> >> > You could always try modifying _PyMem_RawStrdup to return NULL
>> >> >> >> > when
>> >> >> >> > given a
>> >> >> >> > null pointer and see where it then segfaults.
>> >> >> >> >
>> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>> >> >> >> > <chaselton at gmail.com>
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >> Alternatively, is there a hassle-free way to find out what
>> >> >> >> >> changed
>> >> >> >> >> in
>> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>> >> >> >> >> <chaselton at gmail.com>
>> >> >> >> >> wrote:
>> >> >> >> >> > There's a related strdup patch for readline.c, mentioned
>> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>> >> >> >> >> > https://github.com/rave-engine/python3-android/issues/2.
>> >> >> >> >> > There's a patch, but I'm not sure how to modify it for
>> >> >> >> >> > obmalloc.c,
>> >> >> >> >> > as
>> >> >> >> >> > (I think) the functions all belong to Python...they're all
>> >> >> >> >> > prefixed
>> >> >> >> >> > with _PyXx
>> >> >> >> >> >
>> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>> >> >> >> >> > <chaselton at gmail.com>
>> >> >> >> >> > wrote:
>> >> >> >> >> >> Absolutely. Good thing I have addr2line on device
>> >> >> >> >> >>
>> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>> >> >> >> >> >> 0008bbc8
>> >> >> >> >> >> _PyMem_RawStrdup
>> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> >> >> >> >> /bld/python/Python-3.4.2 $
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com>
>> >> >> >> >> >> wrote:
>> >> >> >> >> >>> Could you try the steps at
>> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >> >> >> >> >>> allow you to get a better idea of where libc is crashing.
>> >> >> >> >> >>>
>> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>> >> >> >> >> >>>>
>> >> >> >> >> >>>> Managed to get this out of logcat:
>> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000
>> >> >> >> >> >>>> (code=1),
>> >> >> >> >> >>>> thread
>> >> >> >> >> >>>> 11914 (python) (libc)
>> >> >> >> >> >>>>
>> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc ]
>> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>> >> >> >> >> >>>> 23373
>> >> >> >> >> >>>> (python)
>> >> >> >> >> >>>>
>> >> >> >> >> >>>> Less detail than strace but it seems to be that python is
>> >> >> >> >> >>>> segfaulting
>> >> >> >> >> >>>> libc...
>> >> >> >> >> >>>>
>> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>> >> >> >> >> >>>> <rymg19 at gmail.com>
>> >> >> >> >> >>>> wrote:
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>> On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >> >> >> >> >>>>> <guido at python.org>
>> >> >> >> >> >>>>> wrote:
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> What I see in the strace:
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> ... load libpython3.4m.so.1.0
>> >> >> >> >> >>>>>> ... load libm
>> >> >> >> >> >>>>>> ... open /dev/__properties__ and do something to it
>> >> >> >> >> >>>>>> (what?)
>> >> >> >> >> >>>>>> ... get current time
>> >> >> >> >> >>>>>> ... allocate memory
>> >> >> >> >> >>>>>> ... getuid
>> >> >> >> >> >>>>>> ... segfault
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> That's not a lot to go on, but it doesn't look as if
>> >> >> >> >> >>>>>> it
>> >> >> >> >> >>>>>> has
>> >> >> >> >> >>>>>> started to
>> >> >> >> >> >>>>>> load modules yet.
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> Does /dev/__properties__ ring a bell? Not to me.
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >> >> >> >> >>>>> is the code that works with that file.
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>> This explains it a bit (slides 24-29). Looks like
>> >> >> >> >> >>>>> something
>> >> >> >> >> >>>>> to
>> >> >> >> >> >>>>> do
>> >> >> >> >> >>>>> with
>> >> >> >> >> >>>>> interprocess communication. Likely has nothing to do
>> >> >> >> >> >>>>> with
>> >> >> >> >> >>>>> Python
>> >> >> >> >> >>>>> itself.
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>> Maybe this would be useful?
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> That stack trace would be really helpful.
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >> >> >> >> >>>>>> <chaselton at gmail.com>
>> >> >> >> >> >>>>>> wrote:
>> >> >> >> >> >>>>>>>
>> >> >> >> >> >>>>>>>
>> >> >> >> >> >>>>>>> Apologies...I'm not sure what a stack track is, but I
>> >> >> >> >> >>>>>>> do
>> >> >> >> >> >>>>>>> have
>> >> >> >> >> >>>>>>> the
>> >> >> >> >> >>>>>>> strace. Nearest I can tell, it happens due to an
>> >> >> >> >> >>>>>>> open
>> >> >> >> >> >>>>>>> call,
>> >> >> >> >> >>>>>>> though I
>> >> >> >> >> >>>>>>> am probably wrong.
>> >> >> >> >> >>>>>>> Attaching the strace output to this email. I'm going
>> >> >> >> >> >>>>>>> to
>> >> >> >> >> >>>>>>> head
>> >> >> >> >> >>>>>>> back to
>> >> >> >> >> >>>>>>> the documentation and to back out of some
>> >> >> >> >> >>>>>>> Android-related
>> >> >> >> >> >>>>>>> changes
>> >> >> >> >> >>>>>>> in
>> >> >> >> >> >>>>>>> _localemodule.c
>> >> >> >> >> >>>>>>>
>> >> >> >> >> >>>>>>> On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >> >> >> >> >>>>>>> <guido at python.org>
>> >> >> >> >> >>>>>>> wrote:
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>> There could be a million differences relevant
>> >> >> >> >> >>>>>>>> (unicode,
>> >> >> >> >> >>>>>>>> ints,
>> >> >> >> >> >>>>>>>> ...).
>> >> >> >> >> >>>>>>>> Perhaps
>> >> >> >> >> >>>>>>>> the importlib bootstrap is failing. Perhaps the
>> >> >> >> >> >>>>>>>> dynamic
>> >> >> >> >> >>>>>>>> loading
>> >> >> >> >> >>>>>>>> code
>> >> >> >> >> >>>>>>>> changed. Did you get a stack track? (IIRC strace
>> >> >> >> >> >>>>>>>> shows
>> >> >> >> >> >>>>>>>> a
>> >> >> >> >> >>>>>>>> syscall
>> >> >> >> >> >>>>>>>> trace
>> >> >> >> >> >>>>>>>> --
>> >> >> >> >> >>>>>>>> also useful, but doesn't tell you precisely how
>> >> >> >> >> >>>>>>>> it segfaulted.)
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>> On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >> >> >> >> >>>>>>>> <chaselton at gmail.com>
>> >> >> >> >> >>>>>>>> wrote:
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>> All,
>> >> >> >> >> >>>>>>>>> I recently ditched my attempts to port Python 2.7.8
>> >> >> >> >> >>>>>>>>> to
>> >> >> >> >> >>>>>>>>> Android
>> >> >> >> >> >>>>>>>>> in
>> >> >> >> >> >>>>>>>>> favor of Python 3.4.2. Unfortunately, after using
>> >> >> >> >> >>>>>>>>> the
>> >> >> >> >> >>>>>>>>> same
>> >> >> >> >> >>>>>>>>> configure
>> >> >> >> >> >>>>>>>>> options in the same environment, and modifying the
>> >> >> >> >> >>>>>>>>> setup.py
>> >> >> >> >> >>>>>>>>> as
>> >> >> >> >> >>>>>>>>> needed,
>> >> >> >> >> >>>>>>>>> the newly built binary throws a segfault when the
>> >> >> >> >> >>>>>>>>> generate-posix-vars
>> >> >> >> >> >>>>>>>>> portion of the build is reached...and when it is
>> >> >> >> >> >>>>>>>>> run
>> >> >> >> >> >>>>>>>>> as
>> >> >> >> >> >>>>>>>>> well
>> >> >> >> >> >>>>>>>>> (i.e.
>> >> >> >> >> >>>>>>>>> ./python --help, ./python -E -S -m sysconfig, or
>> >> >> >> >> >>>>>>>>> similar)
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>> I took a strace of ./python, however I'm a bit lost
>> >> >> >> >> >>>>>>>>> when
>> >> >> >> >> >>>>>>>>> reviewing
>> >> >> >> >> >>>>>>>>> it.
>> >> >> >> >> >>>>>>>>> Any ideas as to what may be going on...i.e. why
>> >> >> >> >> >>>>>>>>> Python
>> >> >> >> >> >>>>>>>>> 2.7
>> >> >> >> >> >>>>>>>>> works but
>> >> >> >> >> >>>>>>>>> 3.x throws a segfault?
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>> Thanks in advance,
>> >> >> >> >> >>>>>>>>> Cyd
>> >> >> >> >> >>>>>>>>> ________________________________
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>> Python-Dev mailing list
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>> Python-Dev at python.org
>> >> >> >> >> >>>>>>>>> https://mail.python.org/mailman/listinfo/python-dev
>> >> >> >> >> >>>>>>>>> Unsubscribe:
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>> --
>> >> >> >> >> >>>>>>>> --Guido van Rossum (python.org/~guido)
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> --
>> >> >> >> >> >>>>>> --Guido van Rossum (python.org/~guido)
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> ________________________________
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> Python-Dev mailing list
>> >> >> >> >> >>>>>> Python-Dev at python.org
>> >> >> >> >> >>>>>> https://mail.python.org/mailman/listinfo/python-dev
>> >> >> >> >> >>>>>> Unsubscribe:
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>> --
>> >> >> >> >> >>>>> Ryan
>> >> >> >> >> >>>>> If anybody ever asks me why I prefer C++ to C, my
>> >> >> >> >> >>>>> answer
>> >> >> >> >> >>>>> will
>> >> >> >> >> >>>>> be
>> >> >> >> >> >>>>> simple:
>> >> >> >> >> >>>>> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I
>> >> >> >> >> >>>>> don't
>> >> >> >> >> >>>>> think
>> >> >> >> >> >>>>> that was
>> >> >> >> >> >>>>> nul-terminated."
>> >> >> >> >> >>>>> Personal reality distortion fields are immune to
>> >> >> >> >> >>>>> contradictory
>> >> >> >> >> >>>>> evidence.
>> >> >> >> >> >>>>> -
>> >> >> >> >> >>>>> srean
>> >> >> >> >> >>>>> Check out my website: http://kirbyfan64.github.io/
>> >> >> >> >> >>>
>> >> >> >> >> >>>
>> >> >> >> >> >>> --
>> >> >> >> >> >>> Sent from my Android phone with K-9 Mail. Please excuse my
>> >> >> >> >> >>> brevity.
>> >> >> >> >> >>> Check out my website: http://kirbyfan64.github.io/
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > Ryan
>> >> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will
>> >> >> >> > be
>> >> >> >> > simple:
>> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> >> >> >> > that
>> >> >> >> > was
>> >> >> >> > nul-terminated."
>> >> >> >> > Personal reality distortion fields are immune to contradictory
>> >> >> >> > evidence.
>> >> >> >> > -
>> >> >> >> > srean
>> >> >> >> > Check out my website: http://kirbyfan64.github.io/
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Ryan
>> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
>> >> >> > simple:
>> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> >> >> > that
>> >> >> > was
>> >> >> > nul-terminated."
>> >> >> > Personal reality distortion fields are immune to contradictory
>> >> >> > evidence.
>> >> >> > -
>> >> >> > srean
>> >> >> > Check out my website: http://kirbyfan64.github.io/
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Ryan
>> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
>> >> > simple:
>> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
>> >> > was
>> >> > nul-terminated."
>> >> > Personal reality distortion fields are immune to contradictory
>> >> > evidence.
>> >> > -
>> >> > srean
>> >> > Check out my website: http://kirbyfan64.github.io/
>> >
>> >
>> >
>> >
>> > --
>> > Ryan
>> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>> > nul-terminated."
>> > Personal reality distortion fields are immune to contradictory evidence.
>> > -
>> > srean
>> > Check out my website: http://kirbyfan64.github.io/
>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/
More information about the Python-Dev
mailing list