[Neuroimaging] Thread-safe ArrayProxy/fileslice?
pauldmccarthy at gmail.com
Wed Feb 8 10:48:10 EST 2017
The way that indexed_gzip works, I'd prefer to only have a single file
object in existence - the file object creates an index allowing fast seeks
through the compressed data, and I don't want to have more than one index
I've realised that I need to protect the individual paired calls to seek/
read in fileslice.read_segments (i.e. protect reading of each segment).
Protecting an entire call to read_segments blocks other threads
unacceptably long (e.g. I would like the user to be able to interact with
the current 3D volume while the 4D time series for one or more voxels is
being read in).
So I will add a Lock object to my file objects, and monkey-patch
read_segments to use it.
On 8 February 2017 at 14:20, Christopher Markiewicz <effigies at bu.edu> wrote:
> Is there any reason not to open a separate image object, per-thread? I'd
> think that should keep the state sufficiently separated, though I would be
> interested in the relative performance between that and your solution.
> On Wed, Feb 8, 2017 at 8:55 AM, paul mccarthy <pauldmccarthy at gmail.com>
>> Howdy all,
>> Does anybody have experience accessing image data through the ArrayProxy
>> class (or functions in the fileslice module) in a multi-threaded
>> environment? I am visualising large 4D images which are kept on disk (via
>> my indexed_gzip module), and am having trouble when accessing the data
>> from multiple threads, as the seek/read pairs from different threads
>> will occasionally become intertwined with each other.
>> My hacky workaround is to patch the ArrayProxy.__getitem__ method, and
>> add a threading.Lock to each instance, as follows:
>> import threading
>> import nibabel as nib
>> import nibabel.arrayproxy as ap
>> def ArrayProxy__getitem__(self, slc):
>> if not hasattr(self, '_thread_lock'):
>> return self.__real_getitem__(slc)
>> return ap.ArrayProxy.__real_getitem__(self, slc)
>> # Patch ArrayProxy.__getitem__
>> ap.ArrayProxy.__real_getitem__ = ap.ArrayProxy.__getitem__
>> ap.ArrayProxy.__getitem__ = ArrayProxy__getitem__
>> # Then add a lock to instances
>> # which need to be thread-safe
>> img = nib.load('MNI152_T1_2mm.nii.gz')
>> img.dataobj._thread_lock = threading.Lock()
>> This is the first thing I came up with, although I will probably end up
>> adding the lock to the fileobj, and patching the fileslice.fileslice
>> function instead. Unless there are any better ideas?
>> Neuroimaging mailing list
>> Neuroimaging at python.org
> Neuroimaging mailing list
> Neuroimaging at python.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Neuroimaging