Mailman 3 August 2000 - Python-Dev

The resurrection of comp.lang.python.announce
by bwarsaw＠beopen.com 25 Aug '00

25 Aug '00

Well, after nearly 6 months of inactivity, I'm very happy to say that comp.lang.python.announce is being revived. It will now be moderated by a team of volunteers (see below) using a Mailman mailing list. Details about comp.lang.python.announce, and its mailing list gateway python-announce-list(a)python.org can be found at http://www.python.org/psa/MailingLists.html#clpa Posting guidelines can be found at ftp://rtfm.mit.edu/pub/usenet/comp.lang.python.announce/python-newsgroup-faq This message also serves as a call for moderators. I am looking for 5 experienced Python folks who would like to team moderator the newsgroup. It is a big plus if you've moderated newsgroups before. If you are interested in volunteering, please email me directly. Once I've chosen the current crop of moderators, I'll give you instructions on how to do it. Don't worry if you don't get chosen this time around; I'm sure we'll have some rotation in the moderators ranks as time goes on. Cheers, -Barry

1 0

Re: [Python-checkins] CVS: python/nondist/peps pep-0200.txt,1.28,1.29
by Fred L. Drake, Jr. 25 Aug '00

25 Aug '00

Jeremy Hylton writes: > + Accepted and in progress ... > + * Support for opcode arguments > 2**16 - Charles Waldman > + SF Patch 100893 I checked this in 23 Aug. > + * Range literals - Thomas Wouters > + SF Patch 100902 I thought this was done as well. -Fred -- Fred L. Drake, Jr. <fdrake at beopen.com> BeOpen PythonLabs Team Member

4 3

Re: [Python-Dev] sre and regexp behave baldy under lowmemory conditions
by Jack Jansen 25 Aug '00

25 Aug '00

Recently, Guido van Rossum <guido(a)beopen.com> said: > > test_re wasn't so bad, the only problem was that it crashed with a > > "NULL return without an exception". test_regexp was worse, it crashed > > my machine. > > That's regex, right? regexp was the *really* old regular expression > module we once had. > > Anyway, I don't care about regex, it's old. > > The sre code needs to be robustified, but it's not a high priority for > me. Ok, fine with me. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

1 0

compiling with SSL support on Windows
by Jeremy Hylton 25 Aug '00

25 Aug '00

https://sourceforge.net/bugs/?func=detailbug&bug_id=110683&group_id=5470 We have a bug report about compilation problems in the socketmodule on Windows when using SSL support. Is there any Windows user with OpenSSL who can look into this problem? Jeremy

1 0

Tasks
by Moshe Zadka 25 Aug '00

25 Aug '00

This is a summary of problems I found with the task page: Tasks which I was sure were complete ------------------------------------ 17336 -- Add augmented assignments -- marked 80%. Thomas? 17346 -- Add poll() to selectmodule -- marked 50%. Andrew? Duplicate tasks --------------- 17923 seems to be a duplicate of 17922 -- Moshe Zadka <moshez(a)math.huji.ac.il> There is no IGLU cabal. http://advogato.org/person/moshez

2 1

Re: [Python-checkins] CVS: python/nondist/peps pep-0203.txt,1.11,1.12
by Thomas Wouters 25 Aug '00

25 Aug '00

On Fri, Aug 25, 2000 at 04:11:31AM -0700, Thomas Wouters wrote: > ! [XXX so I am accepting this, but I'm a bit worried about the > ! argument coercion. For x+=y, if x supports augmented assignment, > ! y should only be cast to x's type, not the other way around!] Oh, note that I chose not to do *any* coercion, if x supports the in-place operation. I'm not sure how valuable coercion would be, here, at least not in its current form. (Isn't coercion mostly used by integer types ? And aren't they immutable ? If an in-place method wants to have its argument coerced, it should do so itself, just like with direct method calls.) -- Thomas Wouters <thomas(a)xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

2 1

RE: Passwords after CVS commands
by Tim Peters 25 Aug '00

25 Aug '00

The latest version of Andy Robinson's excellent instructions for setting up a cmdline CVS using SSH under Windows are now available: http://python.sourceforge.net/winssh.txt This is also linked to from the Python-at-SourceForge FAQ: http://python.sourceforge.net/sf-faq.html where it replaces the former "let's try to pretend Windows is Unix(tm)" mish-mash. Riaan Booysen cracked the secret of how to get the Windows ssh-keygen to actually generate keys (ha! don't think I can't hear you Unix(tm) weenies laughing <wink>), and that's the main change from the last version of these instructions I posted here. I added a lot of words to Riann's, admonishing you not to leave the passphrase empty, but so unconvincingly I bet you won't heed my professional advice. and-not-revealing-whether-i-did-ly y'rs - tim

1 0

Re: threading and forking and 2.0 (patch #101226)
by Guido van Rossum 25 Aug '00

25 Aug '00

Here's a patch that Tim & I believe should solve the thread+fork problem properly. I'll try to explain it briefly. I'm not checking this in yet because I need more eyeballs, and because I don't actually have a test to prove that I've fixed the problem. However, our theory is very hopeful. (1) BACKGROUND: A Python lock may be released by a different thread than who aqcuired it, and it may be acquired by the same thread multiple times. A pthread mutex must always be unlocked by the same thread that locked it, and can't be locked more than once. So, a Python lock can't be built out of a simple pthread mutex; instead, a Python lock is built out of a "locked" flag and a <condition variable, mutex> pair. The mutex is locked for at most a few cycles, to protect the flag. This design is Tim's (while still at KSR). (2) PROBLEM: If you fork while another thread holds a mutex, that mutex will never be released, because only the forking thread survives in the child. The LinuxThread manual recommends to use pthread_atfork() to acquire all locks in locking order before the fork, and release them afterwards. A problem with Tim's design here is that even if the forking thread has Python's global interpreter lock, another thread trying to acquire the lock may still hold the mutex at the time of the fork, causing it to be held forever in the child. Charles has posted an effective hack that allocates a new global interpreter lock in the child, but this doesn't solve the problem for other locks. (3) BRAINWAVE: If we use a single mutex shared by all locks, instead of a mutex per lock, we can lock this mutex around the fork and thus prevent any other thread from locking it. This is okay because, while a condition variable always needs a mutex to go with it, there's no rule that the same mutex can't be shared by many condition variables. The code below implements this. (4) MORE WORK: (a) The PyThread API also defines semaphores, which may have a similar problem. But I'm not aware of any use of these (I'm not quite sure why semaphore support was added), so I haven't patched these. (b) The thread_pth.h file define locks in the same way; there may be others too. I haven't touched these. (5) TESTING: Charles Waldman posted this code to reproduce the problem. Unfortunately I haven't had much success with it; it seems to hang even when I apply Charles' patch. import thread import os, sys import time def doit(name): while 1: if os.fork()==0: print name, 'forked', os.getpid() os._exit(0) r = os.wait() for x in range(50): name = 't%s'%x print 'starting', name thread.start_new_thread(doit, (name,)) time.sleep(300) Here's the patch: *** Python/thread_pthread.h 2000/08/23 21:33:05 2.29 --- Python/thread_pthread.h 2000/08/25 04:29:43 *************** *** 84,101 **** * and a <condition, mutex> pair. In general, if the bit can be acquired * instantly, it is, else the pair is used to block the thread until the * bit is cleared. 9 May 1994 tim(a)ksr.com */ typedef struct { char locked; /* 0=unlocked, 1=locked */ /* a <cond, mutex> pair to handle an acquire of a locked lock */ pthread_cond_t lock_released; - pthread_mutex_t mut; } pthread_lock; #define CHECK_STATUS(name) if (status != 0) { perror(name); error = 1; } /* * Initialization. */ --- 84,125 ---- * and a <condition, mutex> pair. In general, if the bit can be acquired * instantly, it is, else the pair is used to block the thread until the * bit is cleared. 9 May 1994 tim(a)ksr.com + * + * MODIFICATION: use a single mutex shared by all locks. + * This should make it easier to cope with fork() while threads exist. + * 24 Aug 2000 {guido,tpeters}(a)beopen.com */ typedef struct { char locked; /* 0=unlocked, 1=locked */ /* a <cond, mutex> pair to handle an acquire of a locked lock */ pthread_cond_t lock_released; } pthread_lock; + static pthread_mutex_t locking_mutex = PTHREAD_MUTEX_INITIALIZER; + #define CHECK_STATUS(name) if (status != 0) { perror(name); error = 1; } /* + * Callbacks for pthread_atfork(). + */ + + static void prefork_callback() + { + pthread_mutex_lock(&locking_mutex); + } + + static void parent_callback() + { + pthread_mutex_unlock(&locking_mutex); + } + + static void child_callback() + { + pthread_mutex_unlock(&locking_mutex); + } + + /* * Initialization. */ *************** *** 113,118 **** --- 137,144 ---- pthread_t thread1; pthread_create(&thread1, NULL, (void *) _noop, &dummy); pthread_join(thread1, NULL); + /* XXX Is the following supported here? */ + pthread_atfork(&prefork_callback, &parent_callback, &child_callback); } #else /* !_HAVE_BSDI */ *************** *** 123,128 **** --- 149,156 ---- #if defined(_AIX) && defined(__GNUC__) pthread_init(); #endif + /* XXX Is the following supported everywhere? */ + pthread_atfork(&prefork_callback, &parent_callback, &child_callback); } #endif /* !_HAVE_BSDI */ *************** *** 260,269 **** if (lock) { lock->locked = 0; - status = pthread_mutex_init(&lock->mut, - pthread_mutexattr_default); - CHECK_STATUS("pthread_mutex_init"); - status = pthread_cond_init(&lock->lock_released, pthread_condattr_default); CHECK_STATUS("pthread_cond_init"); --- 288,293 ---- *************** *** 286,294 **** dprintf(("PyThread_free_lock(%p) called\n", lock)); - status = pthread_mutex_destroy( &thelock->mut ); - CHECK_STATUS("pthread_mutex_destroy"); - status = pthread_cond_destroy( &thelock->lock_released ); CHECK_STATUS("pthread_cond_destroy"); --- 310,315 ---- *************** *** 304,314 **** dprintf(("PyThread_acquire_lock(%p, %d) called\n", lock, waitflag)); ! status = pthread_mutex_lock( &thelock->mut ); CHECK_STATUS("pthread_mutex_lock[1]"); success = thelock->locked == 0; if (success) thelock->locked = 1; ! status = pthread_mutex_unlock( &thelock->mut ); CHECK_STATUS("pthread_mutex_unlock[1]"); if ( !success && waitflag ) { --- 325,335 ---- dprintf(("PyThread_acquire_lock(%p, %d) called\n", lock, waitflag)); ! status = pthread_mutex_lock( &locking_mutex ); CHECK_STATUS("pthread_mutex_lock[1]"); success = thelock->locked == 0; if (success) thelock->locked = 1; ! status = pthread_mutex_unlock( &locking_mutex ); CHECK_STATUS("pthread_mutex_unlock[1]"); if ( !success && waitflag ) { *************** *** 316,330 **** /* mut must be locked by me -- part of the condition * protocol */ ! status = pthread_mutex_lock( &thelock->mut ); CHECK_STATUS("pthread_mutex_lock[2]"); while ( thelock->locked ) { status = pthread_cond_wait(&thelock->lock_released, ! &thelock->mut); CHECK_STATUS("pthread_cond_wait"); } thelock->locked = 1; ! status = pthread_mutex_unlock( &thelock->mut ); CHECK_STATUS("pthread_mutex_unlock[2]"); success = 1; } --- 337,351 ---- /* mut must be locked by me -- part of the condition * protocol */ ! status = pthread_mutex_lock( &locking_mutex ); CHECK_STATUS("pthread_mutex_lock[2]"); while ( thelock->locked ) { status = pthread_cond_wait(&thelock->lock_released, ! &locking_mutex); CHECK_STATUS("pthread_cond_wait"); } thelock->locked = 1; ! status = pthread_mutex_unlock( &locking_mutex ); CHECK_STATUS("pthread_mutex_unlock[2]"); success = 1; } *************** *** 341,352 **** dprintf(("PyThread_release_lock(%p) called\n", lock)); ! status = pthread_mutex_lock( &thelock->mut ); CHECK_STATUS("pthread_mutex_lock[3]"); thelock->locked = 0; ! status = pthread_mutex_unlock( &thelock->mut ); CHECK_STATUS("pthread_mutex_unlock[3]"); /* wake up someone (anyone, if any) waiting on the lock */ --- 362,373 ---- dprintf(("PyThread_release_lock(%p) called\n", lock)); ! status = pthread_mutex_lock( &locking_mutex ); CHECK_STATUS("pthread_mutex_lock[3]"); thelock->locked = 0; ! status = pthread_mutex_unlock( &locking_mutex ); CHECK_STATUS("pthread_mutex_unlock[3]"); /* wake up someone (anyone, if any) waiting on the lock */ --Guido van Rossum (home page: http://www.pythonlabs.com/~guido/)

4 3

checkins not showing on python-checkins and python-patches
by Trent Mick 25 Aug '00

25 Aug '00

Hey all, I recently checked in the Monterey stuff (patch http://sourceforge.net/patch/index.php?func=detailpatch&patch_id=101249&gro… ) but the checkin did not show up on python-checkins and the comment and status change to "Closed" did not show up on python-patches. My checkin was about a full day ago. Is this a potential SourceForge bug? The delay con't be *that* long. Regards, Trent -- Trent Mick TrentM(a)ActiveState.com

3 3

sre and regexp behave baldy under lowmemory conditions
by Jack Jansen 25 Aug '00

25 Aug '00

Both regexp and sre don't behave well under low-memory conditions. I noticed this because test_longexp basically ate all my memory (sigh, I think I'll finally have to give up my private memory allocator and take the 15% performance hit, until I find the time to dig into Vladimir's stuff) so the rest of the regressions tests ran under very tight memory conditions. test_re wasn't so bad, the only problem was that it crashed with a "NULL return without an exception". test_regexp was worse, it crashed my machine. If someone feels the urge maybe they could run the testsuite on unix with a sufficiently low memory-limit. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++

2 1