Mailman 3 September 2000 - Python-Dev

PyWX (Python AOLserver plugin)
by Neil Schemenauer 12 Sep '00

12 Sep '00

You probably want to address the python-dev mailing list. I have CCed this message in the hope that some of the more experienced developers can help. The PyWX website is at: http://pywx.idyll.org/. On Tue, Sep 12, 2000 at 10:40:36AM -0700, Brent Fulgham wrote: > We've run across some problems with the Python's internal threading > design, and its handling of module loading. > > The AOLserver plugin spawns new Python interpreter threads to > service new HTTP connections. Each thread is theoretically its > own interpreter, and should have its own namespace, set of loaded > packages, etc. > > This is largely true, but we run across trouble with the way > the individual threads handle 'argv' variables and current > working directory. > > CGI scripts typically pass data as variables to the script > (as argv). These (unfortunately) are changed globally across > all Python interpreter threads, which can cause problems.... > > In addition, the current working directory is not unique > among independent Python interpreters. So if a script changes > its directory to something, all other running scripts (in > unique python interpreter threads) now have their cwd set to > this directory. > > So we have to address these issues at some point... Any hope > that something like this could be fixed in 2.0? Are you using separate interpreters or one interpreter with multiple threads? It sounds like the latter. If you use the latter than definately things like the process address space and the current working directory are shared across the threads. I don't think I understand your design. Can you explain the architecture of PyWX? Neil

2 1

RE: PyWX (Python AOLserver plugin)
by Brent Fulgham 12 Sep '00

12 Sep '00

> Are you using separate interpreters or one interpreter with multiple > threads? It sounds like the latter. If you use the latter than > definately things like the process address space and the > current working directory are shared across the threads. I don't > think I understand your design. Can you explain the architecture > of PyWX? > There are some documents on the website that give a bit more detail, but in a nutshell we were using the Python interpreter thread concept (Py_InterpreterNew, etc.) to allow 'independent' interpreters to service HTTP requests in the server. We are basically running afoul of the problems with the interpreter isolation, as documented in the various Python embed docs. """Because sub-interpreters (and the main interpreter) are part of the same process, the insulation between them isn't perfect -- for example, using low-level file operations like os.close() they can (accidentally or maliciously) affect each other's open files. Because of the way extensions are shared between (sub-)interpreters, some extensions may not work properly; this is especially likely when the extension makes use of (static) global variables, or when the extension manipulates its module's dictionary after its initialization""" So we are basically stuck. We can't link against Python multiple times, so our only avenue to provide multiple interpreter instances is to use the "Py_InterpreterNew" call and hope for the best. Any hope for better interpreter isolation in 2.0? (2.1?) -Brent

3 2

obsolete urlopen.py in CVS
by Charles G Waldman 11 Sep '00

11 Sep '00

Another obsolete file has magically appeared in my local CVS workspace. I am assuming that I should continue to report these sorts of problems. If not, just tell me and I'll stop with these annoying messages. Is there a mail address for the CVS admin so I don't have to bug the whole list? Lib$ cvs status urlopen.py =================================================================== File: urlopen.py Status: Up-to-date Working revision: 1.7 Repository revision: 1.7 /cvsroot/python/python/dist/src/Lib/Attic/urlopen.py,v Sticky Tag: (none) Sticky Date: (none) Sticky Options: (none)

3 3

Off-topic: common employee IP agreements?
by Greg Ward 11 Sep '00

11 Sep '00

Hi all -- sorry for the off-topic post. I'd like to get a calibration reading from other members of the open source community on an issue that's causing some controversy around here: what sort of employee IP agreements do other software/open source/Python/Linux/Internet-related companies require their employees to sign? I'm especially curious about companies that are prominent in the open source world, like Red Hat, ActiveState, VA Linux, or SuSE; and big companies that are involved in open source, like IBM or HP. I'm also interested in what universities, both around the world and in the U.S., impose on faculty, students, and staff. If you have knowledge -- or direct experience -- with any sort of employee IP agreement, though, I'm curious to hear about it. If possible, I'd like to get my hands on the exact document your employer uses -- precedent is everything! ;-) Thanks -- and please reply to me directly; no need to pollute python-dev with more off-topic posts. Greg -- Greg Ward - software developer gward(a)mems-exchange.org MEMS Exchange / CNRI voice: +1-703-262-5376 Reston, Virginia, USA fax: +1-703-262-5367

1 0

find_recursionlimit.py vs. libpthread vs. linux
by Charles G Waldman 11 Sep '00

11 Sep '00

It has been noted by people doing testing on Linux systems that ulimit -s unlimited python Misc/find_recursionlimit.py will run for a *long* time if you have built Python without threads, but will die after about 2400/2500 iterations if you have built with threads, regardless of the "ulimit" setting. I had thought this was evidence of a bug in Pthreads. In fact (although we still have other reasons to suspect Pthread bugs), the behavior is easily explained. The function "pthread_initialize" in pthread.c contains this very lovely code: /* Play with the stack size limit to make sure that no stack ever grows beyond STACK_SIZE minus two pages (one page for the thread descriptor immediately beyond, and one page to act as a guard page). */ getrlimit(RLIMIT_STACK, &limit); max_stack = STACK_SIZE - 2 * __getpagesize(); if (limit.rlim_cur > max_stack) { limit.rlim_cur = max_stack; setrlimit(RLIMIT_STACK, &limit); } In "internals.h", STACK_SIZE is #defined to (2 * 1024 * 1024) So whenever you're using threads, you have an effective rlimit of 2MB for stack, regardless of what you may *think* you have set via "ulimit -s" One more mystery explained!

1 0

Letting ../site-packages override the standard lib ?!
by M.-A. Lemburg 11 Sep '00

11 Sep '00

As you may have heard, there are problems with the stock XML support and the PyXML project due to both trying to use the xml package namespace (see the xml-sig for details). To provide more flexibility to the third-party tools in such a situation, I think it would be worthwhile moving the site-packages/ entry in sys.path in front of the lib/python2.0/ entry. That way a third party tool can override the standard lib's package or module or take appropriate action to reintegrate the standard lib's package namespace into an extended one. What do you think ? -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

6 6

xml missing in Windows installer?
by Martin v. Loewis 10 Sep '00

10 Sep '00

Using the 2.0b1 Windows installer from BeOpen, I could not find Lib/xml afterwards, whereas the .tgz does contain the xml package. Was this intentional? Did I miss something? Regargds, Martin

5 8

Re: [Pythonmac-SIG] Need some hands to debug MacPython installer
by Jack Jansen 09 Sep '00

09 Sep '00

Oops, indeed some of the new modules were inadvertantly excluded. I'll create a new installer tomorrow (which should also contain the documentation and such). -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

1 0

Need some hands to debug MacPython installer
by Jack Jansen 08 Sep '00

08 Sep '00

Folks, I need some people to test the MacPython 2.0b1 installer. It is almost complete, only things like the readme file and some of the documentation (on building and such) remains to be done. At least: as far as I know. If someone (or someones) could try ftp://ftp.cwi.nl/pub/jack/python/mac/PythonMac20preb1Installer.bin and tell me whether it works that would be much appreciated. One thing to note is that if you've been building 2.0b1 MacPythons from the CVS repository you'll have to remove your preference file first (no such problem with older prefs files). All feedback is welcome, of course, but I'm especially interested in hearing which things I've forgotten (if people could check that expected new modules and such are indeed there), and which bits of the documentation (in Mac:Demo) needs massaging. Oh, and bugs of course, in the unlike event of there being any:-) -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++

3 2

PyObject_SetAttr/GetAttr() and non-string attribute names
by M.-A. Lemburg 08 Sep '00

08 Sep '00

While hacking along on a patch to let set|get|hasattr() accept Unicode attribute names, I found that all current tp_getattro and tp_setattro implementations (classes, instances, methods) expect to find string objects as argument and don't even check for this. Is this documented somewhere ? Should we make the existing implementations aware of other objects as well ? Should we fix the de-facto definition to string attribute names ? My current solution does the latter. It's available as patch on SF. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

1 0