Return str to a callback raise a segfault if used in string formating
Paul Moore
p.f.moore at gmail.com
Fri Oct 13 06:39:12 EDT 2017
As a specific suggestion, I assume the name of the created file is a
string object constructed in the C extension code, somehow. The fact
that you're getting the segfault with some uses of that string
(specifically, passing it to %-formatting) suggests that there's a bug
in the C code that constructs that string. That's where I'd start by
looking. Maybe something isn't zero-terminated that should be? Maybe
your code doesn't set up the character encoding information correctly?
Paul
On 13 October 2017 at 11:15, Thomas Jollans <tjol at tjol.eu> wrote:
> On 2017-10-13 11:07, Vincent Vande Vyvre wrote:
>> Le 13/10/17 à 09:23, Chris Angelico a écrit :
>>> On Fri, Oct 13, 2017 at 4:46 PM, Vincent Vande Vyvre
>>> <vincent.vande.vyvre at telenet.be> wrote:
>>>> Simplified code:
>>>>
>>>> ---%<------------------------------------------
>>>> ...
>>>> ---%<------------------------------------------
>>>>
>>>> If I place self.callback() at the end of the func process that doesn't
>>>> change anything.
>>> First off, exactly what version of Python are you running this under?
>>> With a segfault, you need to be pretty specific - platform, version,
>>> word size, as much as you can gather.
>>>
>>> Secondly: Can you create a version of this that doesn't comment out
>>> part of the work? If you can make a script where, any time you run it,
>>> Python segfaults, that would be extremely helpful.
>>>
>>> Your code currently looks a bit weird. You create a thread, start it,
>>> and then immediately wait for it (join()). When you add a task, if
>>> it's the first task you've added, you process tasks, thus removing
>>> that task. So the class isn't actually doing anything, and logically,
>>> you could simply process the files directly in the loop. I'm guessing
>>> that something in there (probably the threading) is causing the crash,
>>> but without a complete and testable demo, it's hard to be sure.
>>>
>>> ChrisA
>>
>> I'm using 3.6.1 in venv
>>
>> It's not easy to write a runnable code because in this example the
>> method process() is too simplified and the code don't reproduce the
>> segfault.
>
> That is what you'll have to do, though.
> If you strip out as much as possible to create a minimal, working,
> complete example, you will have a much easier time reasoning about what
> is happening. This will also allow other people to reproduce the
> problem, and help.
>
>> The code is an image processing in two parts: the processing himself in
>> CPython and an api in Python.
>
> If you have custom C code, it's likely that there is a memory management
> problem there. It's not unusual for incorrect memory management to cause
> problems in a completely different part of the code, where something
> tries to access freed memory or something.
>
> If removing a call to C fixes it, I recommend you carefully check the
> memory management logic of your C function.
>
>
>>
>> I have written the two parts.
>>
>> In the real code this is the equivalent of the method process():
>> ----------------------------------------------
>> def unraw(self, index=0, dest=""):
>> """Run the demosaication process.
>>
>> Args:
>> index -- the index of the image or "all" if there's more than
>> one image
>> into the file
>> dest -- the absolute file name for the image decoded. If a file
>> with
>> the same name already exists, it will be overwritten.
>>
>> Raise IndexError if index >= self.image_count
>> """
>> if not self.is_raw:
>> fname = os.path.basename(self.filename)
>> raise TypeError("RAW file %s not supported!" % fname)
>>
>> if index >= self.data["image_count"]:
>> raise IndexError("Index of image %s out of range(%s)"
>> %(index, self.data["image_count"]))
>>
>> multi = 0
>> if index == "all":
>> multi = 1
>> index = 0
>>
>> if not dest:
>> dest = self.filename
>>
>> target = os.path.splitext(dest)[0]
>> res = self.raw.demosaicate(index, multi, target) # This call
>> the CPython code
>> self.out_filename = self.raw.out_file
>>
>> def run_multitasks(self):
>> def process(*args):
>> fname, idx, oname = args
>> self.identify()
>> self.unraw(idx, oname)
>>
>> while self.tasks:
>> t = Thread(target=process, args=self.tasks.pop(0))
>> t.start()
>> t.join()
>> if self.callback:
>> self.callback(self.out_filename)
>> ----------------------------------------------------------------
>>
>> I use a list of 20 files and all files are converted, one by one,
>> because the CPython part is not thread safe.
>>
>> But the real interesting thing is the behaviour in the slot() function.
>>
>> Examples:
>> 1 -------------------------------
>> def slot(name):
>> if os.path.isfile(name):
>> print("Created:", name, type(name))
>> print("%s created" % name)
>>
>> result:
>> add: /home/vincent/Images/RAW_orig/RAW_CANON_1DSM2.CR2
>> Scaling with darkness 127, saturation 3712, and
>> multipliers 2.310656 1.000000 1.257116 1.000000
>> AHD interpolation...
>> Converting to sRGB colorspace...
>> Created:
>> /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff
>> <class 'str'>
>> Erreur de segmentation (core dumped)
>> ----------------------------------
>>
>> 2 --------------------------------
>> def slot(name):
>> new = name[:]
>> if os.path.isfile(name):
>> print("Created:", name, type(name))
>> print("New:", new)
>> print("%s created" % new)
>>
>> result:
>> add /home/vincent/Images/RAW_orig/RAW_CANON_1DSM2.CR2
>> Scaling with darkness 127, saturation 3712, and
>> multipliers 2.310656 1.000000 1.257116 1.000000
>> AHD interpolation...
>> Converting to sRGB colorspace...
>> Created:
>> /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff
>> <class 'str'>
>> New: /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff
>> Erreur de segmentation (core dumped)
>> ----------------------------------
>>
>> The most important, of course is the name of the new file created and I
>> can be satisfied with that, I just need the file name, but it's not a
>> reason to ignore the problem.
>>
>> Vincent
>>
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
More information about the Python-list
mailing list