[Patches] [ python-Patches-622537 ] dummy_thread.py implementation
noreply@sourceforge.net
noreply@sourceforge.net
Tue, 10 Dec 2002 18:52:02 -0800
Patches item #622537, was opened at 2002-10-13 00:07
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=622537&group_id=5470
Category: Library (Lib)
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Brett Cannon (bcannon)
Assigned to: Nobody/Anonymous (nobody)
Summary: dummy_thread.py implementation
Initial Comment:
As mentioned in Zack Weinberg's Python-dev thread, "A
few lessons from the tempfile.py rewrite", it was
brought up that not having thread in all distributions
is a pain because you have to write around instances of
it not existing. Guido suggested the compromise of
having a dummy_thread.py so that one could do:
try:
import thread as _thread
except ImportError:
import dummy_thread as _thread
This is specific comment can be found at
http://mail.python.org/pipermail/python-dev/2002-August/027985.html
.
Well, I decided to implement dummy_thread.py . I have
tested it against test_thread.py and test_threading.py
by editing the files locally to import dummy_thread as
thread. All tests did pass.
The only thing that is not necessarily straight-forward
in the implementation is that for non-blocking
lock.acquire() calls, the code actually checks to see
if the lock is available; blocking calls just
automatically acquire the lock since blocking when
there is not real threading would not big good. =) It
was also to appease the assert demons in threading.py
for Condition objects since Condition._is_owned() did
not like having lock.acquire() always succeed.
As for adding regression tests, one could just run
test_thread.py specifically against dummy_thread.py .
The tougher test, though, was getting test_threading.py
to work with threading.py importing dummy_thread.
I am not, as of this exact moment, uploading any
patches to add regression tests since I wrote this on a
whim and I am leaving for vacation on 2002-10-14. I
will be back by November 1, though, and I can try to
come up with how to integrate regression tests then if
no one has beaten me to it.
----------------------------------------------------------------------
>Comment By: Brett Cannon (bcannon)
Date: 2002-12-10 18:52
Message:
Logged In: YES
user_id=357491
Files dummy_thread.zip and dummy_threading.zip (use the
"2002-12-10b" version of dummy_threading.zip!) have been
uploaded. I followed Martin's advise and created zip files
(using ``zipfile.ZipFile``, uncompressed) to be unzipped
from the ``python/dist/src`` directory of your copy of CVS.
Everything should unzip to their proper places from there
(used relative directories from ``src``). The diffs are
``cvs diff -u``, so that ``-p`` stuff Martin was mentioning
should not be an issue. The diffs are unpacked into ``src``
and are obviously named. Hopefully I didn't botch anything. =)
Compared to the old files I cleaned up
``dummy_threading.py`` by removing the redundant assigning I
was doing and I made sure I cleaned up the module's
namespace. I also added ``__all__`` to ``Lib/threading.py``
(initially forgot I had done that which is why there is a
"2002-12-10b" version of dummy_threading.zip).
Can someone please delete all files *except*
dummy_thread.zip ("2002-12-10"), dummy_threading.zip
("2002-12-10b"), and test_thread.py? The reason I want
test_thread.py left is because I think that the current
test_thread.py is not very thorough, and thus I am
disagreeing with Martin by wanting to replace it.
And I still think that ``Lib/pydoc.py`` can be patched to
use ``dummy_thread``, but without a testing suite for that
module I didn't want to risk patching it myself.
----------------------------------------------------------------------
Comment By: Brett Cannon (bcannon)
Date: 2002-12-08 15:07
Message:
Logged In: YES
user_id=357491
This is a follow-up to Martin's question of why I need to
have ``_dummy_threading``. I tried ``from threading import
*`` and ran into the issue of getting an error of trying to
use NoneType as callable. The problem was that I was
deleting ``threading`` from ``sys.modules`` so as to allow
future imports to use ``thread`` and not use the already
imported ``dummy_thread`` version of ``threading`` that I
set up. But with ``threading`` deleted the module code was
being returned as ``None`` objects. And even if I leave it
in and don't delete the module from ``threading`` I still
have issues because subsequent ``threading`` import calls
mess with ``threading`` and I still get the error; I think
it reloads the module and that is not good in this case. So
I don't see how I can get rid of it.
I don't think it is a big issue considering you will only
know about it if you probe ``sys.modules`` and if you are
doing that you had better know what you are doing. Besides,
``re`` imports a total of 3 modules named ``re``, ``sre``,
and ``_sre``, so I don't think having this module add one
more is going to be that big of a deal. =)
----------------------------------------------------------------------
Comment By: Brett Cannon (bcannon)
Date: 2002-12-08 13:31
Message:
Logged In: YES
user_id=357491
In response to Martin's comments:
- The problem with doing it in reverse is that would require
``thread`` to be available for the initial import and that
is not guaranteed like ``dummy_thread`` would be. I could
do some more ``sys.modules`` fiddling but I would like to
keep that to a minimum since it is tricky and error-prone.
If it is not desired to replace the ``thread`` testing suite
then I propose leave it be but still use my new tests for
``dummy_thread``.
- OK. Do realize, though, that I did it this way so as to
allow ``dummy_thread`` to not be held up by
``dummy_threading``. Would it be okay if I did two patch
files, one for ``dummy_thread`` and one for
``dummy_threading``? Considering that all of your comments
are against ``dummy_threading`` (short of the testing suite
issue) I know ``dummy_threading`` is going to be what gets
the most comments and revisions and if this gets dragged out
I would like to at least get ``dummy_thread`` in.
- I assume those are for the docs. Where should I put them?
- Actually ``threading`` is very careful to not pollute the
namespace. I will have a look, though, and see if it is
called for. I will also make sure all of my temp variables
get deleted so as to not pollute ``dummy_threading``'s
namespace.
- Clean up how? The only offensive thing I see are the #
comments but since I am playing around with stuff that
normally shouldn't be I felt ample comments was justified.
If you mean make sure to delete the variables I am going to
add that.
- The original thinking was that I needed a copy of the code
to be referenced from; I wasn't thinking that doing the
``import *`` would make references for ``threading``'s code.
I just quickly tried commenting out all of that code and
did what you suggested and I get an error about NoneType not
being callable. I will look at it some more, but I wouldn't
hold my breath on it working without it.
- You're right, it isn't needed. Once again I was not
thinking in terms of references to modules and how the code
would be pointing to the actual module code and not what was
in ``sys.modules['thread']``.
I will wait about a day or two before I post the next
revision to get more comments.
Thanks for the help, Martin.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2002-12-08 00:48
Message:
Logged In: YES
user_id=21627
A couple of comments:
- please leave the thread tests intact, and have the dummy
tests make use of the thread tests. You should be able to
use the same technique, just reverse.
- please provide the next version of the patch as an archive
file (.zip/.tgz), which unpacks all files into the right
place, and includes a single patch to apply with patch -pX
(ideally for X=0).
- Feel free to use the \moduleauthor and \sectionauthor markups.
- Consider providing threading.__all__ (and copying this
into dummy_threading). Perhaps python-dev needs to comment
on whether this is desirable.
- Clean up holding_*
- What is the purpose of _dummy_threading? Can't you just do
"from threading import *"?
- Why is it necessary to override _start_new_thread? Doesn't
the "import thread" in threading.py already pick up
dummy_threading?
----------------------------------------------------------------------
Comment By: Brett Cannon (bcannon)
Date: 2002-12-07 19:37
Message:
Logged In: YES
user_id=357491
OK, so now both ``dummy_thread`` and ``dummy_threading`` are
done. ``dummy_threading`` was requested by python-dev so as
to not have ``threading`` be imported even if ``thread`` is
not available. It actually uses ``threading`` itself and
through some trickery (read below) just subs out the
``thread`` code and have it use ``dummy_thread``.
Here is the file breakdown:
dummy_thread.py :
Implementation of dummy_thread (I tried to not go overboard
with the name of the file =)
dummy_threading :
I think you know what this is. =) The trick to making this
work was a lot of ``sys.modules`` manipulation. It adds an
extra module named ``_dummy_threading``; I figured naming it
this would be okay since other modules like ``re`` have
other modules with a similar naming scheme.
added_dummy.diff :
Patches Lib/Queue.py and Lib/tempfile.py to use
``dummy_thread``. As I have mentioned in another post,
``pydoc`` probably can be safely patched as well, but since
there are no tests I don't want to risk it.
doc_dummy.diff :
Patches Doc/lib/libthread.text and Doc/lib/libthreading.tex
to mention their corresponding dummy twins.
libdummythread.tex :
Documentation for ``dummy_thread``.
libdummythreading.tex :
Documentation fo ``dummy_threading``.
test_dummy_thread.py :
I think it is obvious from the name.
test_dummy_threading.py :
Ditto for this file as well. It is basically just
``test_threading`` with a ``test_main()`` function instead
of being run on import. I think a Lib/test/output/ file
needs to be created (should be able to just copy
``test_threading``'s file).
test_thread.py :
I wrote the tests for ``dummy_thread`` as if I was writing
tests for ``thread``. Since I went to that trouble I made
sure that ``test_dummy_thread`` could be run while using
``thread`` instead of ``dummy_thread``. This testing suite
is much more in-depth than the one currently in the stdlib.
I wasn't able to pull this off for ``test_threading``.
I think the docs and ``dummy_threading`` are the biggest
things to look at and check. ``dummy_threading`` because of
the amount of ``sys.modules`` fiddling it does (although I
tested it rather thoroughly to make sure nothing got messed
up after the file is imported). The docs just because they
are my first attempt and they are rather thin (although that
might be okay).
``dummy_thread`` is completely independent and can be added
without ``dummy_threading``, so if there is any hold-ups
with ``dummy_threading``, please at least get
``dummy_thread`` in.
----------------------------------------------------------------------
Comment By: Tim Peters (tim_one)
Date: 2002-12-04 18:01
Message:
Logged In: YES
user_id=31435
Deleted all attachments, at Brett's request.
----------------------------------------------------------------------
Comment By: Brett Cannon (bcannon)
Date: 2002-11-27 19:07
Message:
Logged In: YES
user_id=357491
So all patches and files needed for dummy_thread to be
accepted into the stdlib are now uploaded. The various
parts of this patch (which all don't need to be accepted
which is why I did it the way I have) are as follows:
The module:
dummy_thread.py is the module. test_dummy_thread.py is the
testing suite.
New test files:
Since I had to basically simulate threads, I wrote the tests
as if I was going to test the thread module. So I wrote
test_dummy_thread.py in such a way that ``test_main()`` can
accept a module to use as the thread implementation. The
included test_thread.py uses test_dummy_thread.py to test
the thread module. It is more in-depth then the current
implementation. If my version of test_thread.py is used,
then Lib/test/output/test_thread will need to be removed (my
testing suite uses PyUnit). As an aside, dummy_thread.py
does pass the current test_thread.py testing suite.
Changed modules:
added_dummy.diff is a diff that changes Queue, threading,
and tempfile to use dummy_thread when thread is not
available. They all still pass their tests when using
dummy_thread (dummy_thread has one design decision just to
pass threading's ``assert`` statements; all info is in the
doc strings). pydoc can be changed so as to not use
threading if the thread module is not available, but there
are no tests for the module, so I will let the person who
checks this in make the call whether it should be added. I
tried it and everything seemed to work fine so I personally
think it is safe for pydoc to stay as it is.
Documentation:
libdummythread.tex is my attempt at writing the docs for
dummy_thread. Since it basically mirrors thread, I state to
use thread's docs for API info and explain what
dummy_thread is for. doc_dummy.tex adds notes to
libthread.tex and libthreading.tex mentioning dummy_thread
and in threading's case how to make sure that you only
import threading if you want only true threading.
Mentioning Queue's use of dummy_thread as backup could be
useful, but since there is no lock-up problems I don't see
the need.
dummy_thread has only one issue with replacing thread in all
cases: blocking I/O. Both test_asynchat and test_socket
block on their threads because they spawn both server and
client threads that block waiting for each other. But both
modules import thread explicitly, so there is no worry with
these modules running with dummy_thread as their thread
implementation. I also couldn't run test_socketserver
because I can't run test_normalization.
But other then those three tests, all other tests pass both
under a regular debug compile and a --without-thread
compile; things just run a little slower. =)
The 2002-10-13 copy of dummy_thread.py can be deleted (wish
SF would let the patch creator delete files).
This is my first big patch that touches docs, test, stdlib
modules and has a new doc file. Just thought I should warn
you. =)
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=622537&group_id=5470