[IronPython] Problems with hashlib and threading

[Curt Hagenlocher]

Not without looking at the source code...

We want to be at least as thread-safe as CPython, with the exception that we
have no interest in implementing the equivalent of a "global interpreter
lock" :).  This particular case is just bad coding on our part.  I can't
promise you that there aren't other examples, but at least this one is
pretty easy to fix.

On Sun, Jan 4, 2009 at 8:58 AM, Brian Merrell <brian at merrells.org> wrote:

> Thanks Curt.  I've filed a bug report.  It makes me a bit nervous realizing
> that it isn't entirely transparent when shared memory/objects are being
> used.  Normally I would think a local instance of hashlib.md5() would be
> thread-safe from other local instances.  Is there any documentation of what
> other modules use global/static instances or other non-thread-safe
> implementations in IronPython?
>
>
> On Tue, Dec 30, 2008 at 12:13 PM, Curt Hagenlocher <curt at hagenlocher.org>wrote:
>
>> I'm going to guess that this is because we use a static instance
>> of MD5CryptoServiceProvider to do the hashing, something which may not be
>> thread-safe.  It looks like all of the IronPython crypto implementations do
>> the same thing, so the SHA functions may have a similar issue.
>> Please file this as a bug on Codeplex.
>>
>> On Sat, Dec 20, 2008 at 1:56 PM, Brian Merrell <brian at merrells.org>wrote:
>>
>>> I get the following error periodically:
>>>
>>> Exception in thread Thread-2:
>>> Traceback (most recent call last):
>>>   File "C:\Program Files\IronPython 2.0\Lib\threading.py", line 486, in
>>> _Thread__bootstrap_inner
>>>     self.run()
>>>   File "initial_ingest.py", line 33, in run
>>>     md5 = hashlib.md5(contents).hexdigest()
>>> SystemError: Hash not valid for use in specified state.
>>>
>>> I am not sharing a md5() object between threads and have tried to avoid
>>> any shared memory issues:
>>>
>>>
>>> class Convert(threading.Thread):
>>>     def __init__(self, queue):
>>>         threading.Thread.__init__(self)
>>>         self.queue = queue
>>>     def run(self):
>>>         import hashlib
>>>         while True:
>>>             abs_filename = self.queue.get()
>>>             f = open(abs_filename, "rb")
>>>             contents = f.read()
>>>             f.close()
>>>             md5 = hashlib.md5(contents).hexdigest()
>>>             print md5, abs_filename
>>>             self.queue.task_done()
>>>
>>> Any ideas why this is failing?  Thanks,
>>>
>>> -brian
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at lists.ironpython.com
>>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>>
>>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.ironpython.com
>> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>>
>>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20090104/21870ad7/attachment.html>