Mailman 3 July 1999 - Python-announce-list

Alfajor - Cookie Filter
by Andrew Cooke 11 Jul '99

11 Jul '99

Hi, Alfajor is an http cookie filter, written in Python with an optional GUI. It acts as an http proxy (you must configure your browser to use it) and can either contact sites directly or work with a second proxy (eg. a cache). The program allows cookies to be sent from your browser to any address which matches a list of regular expressions. The GUI allows the filter to be turned on and off, sites to be added or removed as they are accessed, and the list of regular expressions to be edited. The program is (c) Andrew Cooke 1999 and released under the GPL. For more info, please look at http://www.andrewcooke.free-online.co.uk/jara/alfajor/index.html Thanks, Andrew Sent via Deja.com http://www.deja.com/ Share what you know. Learn what you don't. <A HREF="http://www.andrewcooke.free-online.co.uk/jara/alfajor/index.html">Alfajor</A> - configurable HTTP cookie filter written in Python. (10-Jul-99) -- ----------- comp.lang.python.announce (moderated) ---------- Article Submission Address: python-announce(a)python.org Python Language Home Page: http://www.python.org/ Python Quick Help Index: http://www.python.org/Help.html ------------------------------------------------------------

1 0

Signature 0.1
by neelk＠neelk 10 Jul '99

10 Jul '99

Signature.py is a module that makes reflecting on the function call signatures of callable objects a lot easier. A common question in Python is "How do I ask the interpreter what arguments a function takes?" and while the information is obviously available, I haven't found any simple, high-level interfaces to it. That's why I wrote Signature -- it's useful enough that it's made it into my personal library of frequently-used utilities. You can find it at: http://www.sff.net/people/neelk/free-software/Signature.py Caveats/TODO: o I don't know if it will work on JPython -- I exploited a lot of undocumented object attributes to figure out what the call signature is, and I have no idea if the same data is available there. o I haven't figured out how to look into C builtins yet -- anyone who can teach me how to do this from within Python will earn my undying gratitude. :) Example of use: >>> def foo(x, y, z=-1.0, *args, **kw): ... return (x+y)**z ... >>> f = Signature(foo) >>> print 'ordinary arglist:', f.ordinary_args() ordinary arglist: ('x', 'y', 'z') >>> print 'special_args:', f.special_args() special_args: {'keyword': 'kw', 'positional': 'args'} >>> print 'full arglist:', f.full_arglist() full arglist: ['x', 'y', 'z', 'args', 'kw'] >>> print 'defaults:', f.defaults() defaults: {'z': -1.0} >>> print 'signature:', str(f) signature: foo(x, y, z=-1.0, *args, **kw) Neel <A HREF="http://www.sff.net/people/neelk/free-software/Signature.py"> Signature 0.1</A> - for reflecting on the function call signatures of callable objects. (09-Jul-99) -- ----------- comp.lang.python.announce (moderated) ---------- Article Submission Address: python-announce(a)python.org Python Language Home Page: http://www.python.org/ Python Quick Help Index: http://www.python.org/Help.html ------------------------------------------------------------

1 0

Signature 0.1
by neelk＠neelk 10 Jul '99

10 Jul '99

Signature.py is a module that makes reflecting on the function call signatures of callable objects a lot easier. A common question in Python is "How do I ask the interpreter what arguments a function takes?" and while the information is obviously available, I haven't found any simple, high-level interfaces to it. That's why I wrote Signature -- it's useful enough that it's made it into my personal library of frequently-used utilities. You can find it at: http://www.sff.net/people/neelk/free-software/Signature.py Caveats/TODO: o I don't know if it will work on JPython -- I exploited a lot of undocumented object attributes to figure out what the call signature is, and I have no idea if the same data is available there. o I haven't figured out how to look into C builtins yet -- anyone who can teach me how to do this from within Python will earn my undying gratitude. :) Example of use: >>> def foo(x, y, z=-1.0, *args, **kw): ... return (x+y)**z ... >>> f = Signature(foo) >>> print 'ordinary arglist:', f.ordinary_args() ordinary arglist: ('x', 'y', 'z') >>> print 'special_args:', f.special_args() special_args: {'keyword': 'kw', 'positional': 'args'} >>> print 'full arglist:', f.full_arglist() full arglist: ['x', 'y', 'z', 'args', 'kw'] >>> print 'defaults:', f.defaults() defaults: {'z': -1.0} >>> print 'signature:', str(f) signature: foo(x, y, z=-1.0, *args, **kw) Neel <A HREF="http://www.sff.net/people/neelk/free-software/Signature.py"> Signature 0.1</A> - for reflecting on the function call signatures of callable objects. (09-Jul-99) -- ----------- comp.lang.python.announce (moderated) ---------- Article Submission Address: python-announce(a)python.org Python Language Home Page: http://www.python.org/ Python Quick Help Index: http://www.python.org/Help.html ------------------------------------------------------------

1 0

[fm] gimp-python 0.4
by Markus Fleck 10 Jul '99

10 Jul '99

gimp-python 0.4 james - July 05th 1999, 12:13 EST Gimp-Python is a set of python modules that allow you to write plugins for the GIMP in python. You get the best of both worlds, with all the power of the C plugin interface available, and the automatic GUI generation of Script-Fu. This plugin is also good if you don't like programming in scheme, perl or tcl. Changes: Added gimp-1.1 support (it should still work with gimp-1.0), made some incompatible changes to the API (removed the drawable type, and moved all its functionality to the layer and channel types), added a script-fu like interface module that handles all the GUI stuff of a plugin for you -- see the example plugins for details. Download:ftp://ftp.daa.com.au/pub/james/pygimp/ Homepage:http://www.daa.com.au/~james/pygimp/ Author: James Henstridge License: GPL Category: Development/Python Modules Depends on: gimp, python, pygtk Freshmeat (c) 1999 scoop(a)freshmeat.net <A HREF="http://www.daa.com.au/~james/pygimp/">gimp-python 0.4</A> - set of python modules that allow you to write plug-ins for the <A HREF="http://www.gimp.org">GIMP</A> (GNU Image Manipulation Program) in Python. (05-Jul-99) -- ----------- comp.lang.python.announce (moderated) ---------- Article Submission Address: python-announce(a)python.org Python Language Home Page: http://www.python.org/ Python Quick Help Index: http://www.python.org/Help.html ------------------------------------------------------------

1 0

[fm] REalizer 0.1
by Markus Fleck 10 Jul '99

10 Jul '99

REalizer 0.1 Slayne - July 05th 1999, 18:15 EST REalizer is a GUI for entering and testing regular expressions provided by the python re module. Changes: First release. Download:http://www.globalserve.net/~catlee/REalizer.py Author: Chris AtLee License: GPL Category: Development/Tools Depends on: Python Freshmeat (c) 1999 scoop(a)freshmeat.net <A HREF="http://www.globalserve.net/~catlee/REalizer.py">REalizer 0.1</A> - GUI for entering and testing regular expressions provided by the python re module. (05-Jul-99) -- ----------- comp.lang.python.announce (moderated) ---------- Article Submission Address: python-announce(a)python.org Python Language Home Page: http://www.python.org/ Python Quick Help Index: http://www.python.org/Help.html ------------------------------------------------------------

1 0

Zope Weekly News - Wed, 7 Jul 1999
by Amos Latteier 08 Jul '99

08 Jul '99

Hello, It's been a fairly uneventful week in Zopeland. As always lots of people are asking questions and providing good answers. But this week's seen few announcements and software releases, perhaps because of the holiday. Another Zope alpha should be out soon, and that's sure to generate some activity. * Anthony Pfrunder posted an update about his binary distribution of Zope 2.0 alpha for Windows and an upcoming project to make Zope work better with the Python Imaging Library and OpenGL. http://student.uq.edu.au/~s341625 http://www.zope.org/pipermail/zope/1999-July/006298.html * The unstoppable Martijn Pieters posted a patch to allow customizing tree tag decoration. Now longer will you have to use those tired plus and minus boxes. http://www.zope.org/pipermail/zope/1999-July/006401.html * Paul Everitt announced a press release detailing how Digital Creations and UserLand are making Zope and Frontier play nicely together. http://biz.yahoo.com/prnews/990707/tx_digital_1.html * Over on zope-dev list David Jacobs started an interesting discussion of enhancing Zope security features. http://www.zope.org/pipermail/zope-dev/1999-July/000771.html http://www.zope.org/pipermail/zope-dev/1999-July/000773.html * Digital Creations intern Cathi Davey posted documentation to the zope-dev list about some additional type converting facilities she's added to ZPublisher. http://www.zope.org/pipermail/zope-dev/1999-July/000724.html Until next week. -Amos == Amos Latteier mailto:amos@digicool.com Digital Creations http://www.digicool.com -- ----------- comp.lang.python.announce (moderated) ---------- Article Submission Address: python-announce(a)python.org Python Language Home Page: http://www.python.org/ Python Quick Help Index: http://www.python.org/Help.html ------------------------------------------------------------

1 0

Python Language FAQ - Section 8
by Markus Fleck 07 Jul '99

07 Jul '99

This FAQ newsgroup posting has been automatically converted from an HTML snapshot of the original Python FAQ; please refer to the original "Python FAQ Wizard" at <http://grail.cnri.reston.va.us/cgi-bin/faqw.py> if source code snippets given in this document do not work - incidentally some formatting information may have been lost during the conversion. ---------------------------------------------------------------------------- The whole Python FAQ - Section 8 Last changed on Mon Jun 28 19:36:09 1999 EDT (Entries marked with ** were changed within the last 24 hours; entries marked with * were changed within the last 7 days.) ---------------------------------------------------------------------------- 8. Python on Windows 8.1. Using Python for CGI on Microsoft Windows 8.2. How to check for a keypress without blocking? 8.3. $PYTHONPATH 8.4. dedent syntax errors 8.5. How do I emulate os.kill() in Windows? 8.6. Why does os.path.isdir() fail on NT shared directories? 8.7. PyRun_SimpleFile() crashes on Windows but not on Unix 8.8. Import of _tkinter fails on Windows 95/98 8.9. Can't extract the downloaded documentation on Windows 8.10. Can't get Py_RunSimpleFile() to work. 8.11. Where is Freeze for Windows? 8.12. Is a *.pyd file the same as a DLL? 8.13. Missing cw3215mt.dll (or missing cw3215.dll) ---------------------------------------------------------------------------- 8. Python on Windows ---------------------------------------------------------------------------- 8.1. Using Python for CGI on Microsoft Windows Setting up the Microsoft IIS Server/Peer Server: On the Microsoft IIS server or on the Win95 MS Personal Web Server you set up python in the same way that you would set up any other scripting engine. Run regedt32 and go to: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W3SVC\Parameters\ScriptMap and enter the following line (making any specific changes that your system may need) .py :REG_SZ: c:\<path to python>\python.exe -u %s %s This line will allow you to call your script with a simple reference like: http://yourserver/scripts/yourscript.py provided "scripts" is an "executable" directory for your server (which it usually is by default). The "-u" flag specifies unbuffered and binary mode for stdin - needed when working with binary data In addition, it is recommended by people who would know that using ".py" may not be a good idea for the file extensions when used in this context (you might want to reserve *.py for support modules and use *.cgi or *.cgp for "main program" scripts). However, that issue is beyond this Windows FAQ entry. Netscape Servers: Information on this topic exists at: http://home.netscape.com/comprod/server_central/support/fasttrack_man/progr… ---------------------------------------------------------------------------- 8.2. How to check for a keypress without blocking? Use the msvcrt module. This is a standard Windows-specific extensions in Python 1.5 and beyond. It defines a function kbhit() which checks whether a keyboard hit is present; also getch() which gets one character without echo. Plus a few other goodies. (Search for "keypress" to find an answer for Unix as well.) ---------------------------------------------------------------------------- 8.3. $PYTHONPATH In MS-DOS derived environments, a unix variable such as $PYTHONPATH is set as PYTHONPATH, without the dollar sign. PYTHONPATH is useful for specifying the location of library files. ---------------------------------------------------------------------------- 8.4. dedent syntax errors Tim <tim_one(a)email.msn.com> sez: the original content of this FAQ (below) makes little sense. The FAQ does not recommend using tabs, and Guido's Python Style Guide recommends 4 spaces for distributed Python code; this is also the Emacs python-mode default; see http://www.python.org/doc/essays/styleguide.html Under any editor mixing tabs and spaces is a bad idea. MSVC is no different in this respect, and is easily configured to use spaces: Take Tools -> Options -> Tabs, and for file type "Default" set "Tab size" and "Indent size" to 4, and select the "Insert spaces" radio button. If you suspect mixed tabs and spaces are causing problems in leading whitespace, run Python with the -t switch or, run Tools/Scripts/tabnanny.py to check a directory tree in batch mode. [original follows] The FAQ really means it when it suggests using tabs, not spaces, for indentation control. This may be a bigger problem in windows than in unix. For instance, the Microsoft Visual C++ programmers editor, in its default configuration, does not indicate whether the white space is spaces or tabs. If it is spaces, you are likely to get syntax errors not obviously related to the indentation. ---------------------------------------------------------------------------- 8.5. How do I emulate os.kill() in Windows? Use win32api: def kill(pid): """kill function for Win32""" import win32api handle = win32api.OpenProcess(1, 0, pid) return (0 != win32api.TerminateProcess(handle, 0)) ---------------------------------------------------------------------------- 8.6. Why does os.path.isdir() fail on NT shared directories? The solution appears to be always append the "\\" on the end of shared drives. >>> import os >>> os.path.isdir( '\\\\rorschach\\public') 0 >>> os.path.isdir( '\\\\rorschach\\public\\') 1 [Blake Winton responds:] I've had the same problem doing "Start >> Run" and then a directory on a shared drive. If I use "\\rorschach\public", it will fail, but if I use "\\rorschach\public\", it will work. For that matter, os.stat() does the same thing (well, it gives an error for "\\\\rorschach\\public", but you get the idea)... I've got a theory about why this happens, but it's only a theory. NT knows the difference between shared directories, and regular directories. "\\rorschach\public" isn't a directory, it's _really_ an IPC abstraction. This is sort of lended credence to by the fact that when you're mapping a network drive, you can't map "\\rorschach\public\utils", but only "\\rorschach\public". [Clarification by funkster(a)midwinter.com] It's not actually a Python question, as Python is working just fine; it's clearing up something a bit muddled about Windows networked drives. It helps to think of share points as being like drive letters. Example: k: is not a directory k:\ is a directory k:\media is a directory k:\media\ is not a directory The same rules apply if you substitute "k:" with "\\conky\foo": \\conky\foo is not a directory \\conky\foo\ is a directory \\conky\foo\media is a directory \\conky\foo\media\ is not a directory ---------------------------------------------------------------------------- 8.7. PyRun_SimpleFile() crashes on Windows but not on Unix I've seen a number of reports of PyRun_SimpleFile() failing in a Windows port of an application embedding Python that worked fine on Unix. PyRun_SimpleString() works fine on both platforms. I think this happens because the application was compiled with a different set of compiler flags than Python15.DLL. It seems that some compiler flags affect the standard I/O library in such a way that using different flags makes calls fail. You need to set it for multi-threaded DLL. ---------------------------------------------------------------------------- 8.8. Import of _tkinter fails on Windows 95/98 Sometimes, the import of _tkinter fails on Windows 95 or 98, complaining with a message like the following: ImportError: DLL load failed: One of the library files needed to run this application cannot be found. It could be that you haven't installed Tcl/Tk, but if you did install Tcl/Tk, and the Wish application works correctly, the problem may be that its installer didn't manage to edit the autoexec.bat file correctly. It tries to add a statement that changes the PATH environment variable to include the Tcl/Tk 'bin' subdirectory, but sometimes this edit doesn't quite work. Opening it with notepad usually reveals what the problem is. (One additional hint, noted by David Szafranski: you can't use long filenames here; e.g. use C:\PROGRA~1\Tcl\bin instead of C:\Program Files\Tcl\bin.) ---------------------------------------------------------------------------- 8.9. Can't extract the downloaded documentation on Windows Sometimes, when you download the documentation package to a Windows machine using a web browser, the file extension of the saved file ends up being .EXE. This is a mistake; the extension should be .TGZ. Simply rename the downloaded file to have the .TGZ extension, and WinZip will be able to handle it. (If your copy of WinZip doesn't, get a newer one from http://www.winzip.com.) ---------------------------------------------------------------------------- 8.10. Can't get Py_RunSimpleFile() to work. This is very sensitive to the compiler vendor, version and (perhaps) even options. If the FILE* structure in your embedding program isn't the same as is assumed by the Python interpreter it won't work. The Python 1.5.* DLLs (python15.dll) are all compiled with MS VC++ 5.0 and with multithreading-DLL options (/MD, I think). If you can't change compilers or flags, try using Py_RunSimpleString(). A trick to get it to run an arbitrary file is to construct a call to execfile() with the name of your file as argument. ---------------------------------------------------------------------------- 8.11. Where is Freeze for Windows? ("Freeze" is a program that allows you to ship a Python program as a single stand-alone executable file. It is not a compiler, your programs don't run any faster, but they are more easily distributable (to platforms with the same OS and CPU). Read the README file of the freeze program for more disclaimers.) You can use freeze on Windows, but you must download the source tree (see http://www.python.org/download/download_source.html). This is recommended for Python 1.5.2 (and betas thereof) only; older versions don't quite work. You need the Microsoft VC++ 5.0 compiler (maybe it works with 6.0 too). You probably need to build Python -- the project files are all in the PCbuild directory. The freeze program is in the Tools\freeze subdirectory of the source tree. ---------------------------------------------------------------------------- 8.12. Is a *.pyd file the same as a DLL? Yes, .pyd files are dll's. But there are a few differences. If you have a DLL named foo.pyd, then it must have a function initfoo(). You can then write Python "import foo", and Python will search for foo.pyd (as well as foo.py, foo.pyc) and if it finds it, will attempt to call initfoo() to initialize it. You do not link your .exe with foo.lib, as that would cause Windows to require the DLL to be present. Note that the search path for foo.pyc is PYTHONPATH, not the same as the path that Windows uses to search for foo.dll. Also, foo.pyd need not be present to run your program, whereas if you linked your program with a dll, the dll is required. Of course, foo.pyd is required if you want to say "import foo". In a dll, linkage is declared in the source code with __declspec(dllexport). In a .pyd, linkage is defined in a list of available functions. ---------------------------------------------------------------------------- 8.13. Missing cw3215mt.dll (or missing cw3215.dll) Sometimes, when using Tkinter on Windows, you get an error that cw3215mt.dll or cw3215.dll is missing. Cause: you have an old Tcl/Tk DLL built with cygwin in your path (probably C:\Windows). You must use the Tcl/Tk DLLs from the standard Tcl/Tk installation (Python 1.5.2 comes with one). ---------------------------------------------------------------------------- -- ----------- comp.lang.python.announce (moderated) ---------- Article Submission Address: python-announce(a)python.org Python Language Home Page: http://www.python.org/ Python Quick Help Index: http://www.python.org/Help.html ------------------------------------------------------------

1 0

Python Language FAQ - Section 7
by Markus Fleck 07 Jul '99

07 Jul '99

This FAQ newsgroup posting has been automatically converted from an HTML snapshot of the original Python FAQ; please refer to the original "Python FAQ Wizard" at <http://grail.cnri.reston.va.us/cgi-bin/faqw.py> if source code snippets given in this document do not work - incidentally some formatting information may have been lost during the conversion. ---------------------------------------------------------------------------- The whole Python FAQ - Section 7 Last changed on Mon Jun 28 19:36:09 1999 EDT (Entries marked with ** were changed within the last 24 hours; entries marked with * were changed within the last 7 days.) ---------------------------------------------------------------------------- 7. Using Python on non-UNIX platforms 7.1. Is there a Mac version of Python? 7.2. Are there DOS and Windows versions of Python? 7.3. Is there an OS/2 version of Python? 7.4. Is there a VMS version of Python? 7.5. What about IBM mainframes, or other non-UNIX platforms? 7.6. Where are the source or Makefiles for the non-UNIX versions? 7.7. What is the status and support for the non-UNIX versions? 7.8. I have a PC version but it appears to be only a binary. Where's the library? 7.9. Where's the documentation for the Mac or PC version? 7.10. How do I create a Python program file on the Mac or PC? 7.11. How can I use Tkinter on Windows 95/NT? 7.12. cgi.py (or other CGI programming) doesn't work sometimes on NT or win95! 7.13. Why doesn't os.popen() work in PythonWin on NT? 7.14. How do I use different functionality on different platforms with the same program? 7.15. Is there an Amiga version of Python? 7.16. Why doesn't os.popen()/win32pipe.popen() work on Win9x? ---------------------------------------------------------------------------- 7. Using Python on non-UNIX platforms ---------------------------------------------------------------------------- 7.1. Is there a Mac version of Python? Yes, see the "mac" subdirectory of the distribution sites, e.g. ftp://ftp.python.org/pub/python/mac/. ---------------------------------------------------------------------------- 7.2. Are there DOS and Windows versions of Python? Yes. The core windows binaries are available from http://www.python.org/windows/. There is a plethora of Windows extensions available, including a large number of not-always-compatible GUI toolkits. The core binaries include the standard Tkinter GUI extension. Most windows extensions can be found (or referenced) at http://www.python.org/windows/ Windows 3.1/DOS support seems to have dropped off recently. You may need to settle for an old version of Python one these platforms. One such port is WPY WPY: Ports to DOS, Windows 3.1(1), Windows 95, Windows NT and OS/2. Also contains a GUI package that offers portability between Windows (not DOS) and Unix, and native look and feel on both. ftp://ftp.python.org/pub/python/wpy/. ---------------------------------------------------------------------------- 7.3. Is there an OS/2 version of Python? Yes, see the "pc" and "wpy" subdirectory of the distribution sites (see above). ---------------------------------------------------------------------------- 7.4. Is there a VMS version of Python? Yes, there is a port of Python 1.4 to OpenVMS and a few ports of 1.2 to VMS. See ftp://ftp.python.org/pub/python/contrib/Porting/vms/. Uwe Zessin has ported Python 1.5.x to OpenVMS. See http://decus.decus.de/~zessin/. ---------------------------------------------------------------------------- 7.5. What about IBM mainframes, or other non-UNIX platforms? I haven't heard about these, except I remember hearing about an OS/9 port and a port to Vxworks (both operating systems for embedded systems). If you're interested in any of this, go directly to the newsgroup and ask there, you may find exactly what you need. For example, a port to MPE/iX 5.0 on HP3000 computers was just announced, see http://www.allegro.com/software/. ---------------------------------------------------------------------------- 7.6. Where are the source or Makefiles for the non-UNIX versions? The standard sources can (almost) be used. Additional sources can be found in the platform-specific subdirectories of the distribution. ---------------------------------------------------------------------------- 7.7. What is the status and support for the non-UNIX versions? I don't have access to most of these platforms, so in general I am dependent on material submitted by volunteers. However I strive to integrate all changes needed to get it to compile on a particular platform back into the standard sources, so porting of the next version to the various non-UNIX platforms should be easy. ---------------------------------------------------------------------------- 7.8. I have a PC version but it appears to be only a binary. Where's the library? If you are running any version of Windows, then you have the wrong distribution. The FAQ lists current Windows versions. Notably, Pythonwin and wpy provide fully functional installations. But if you are sure you have the only distribution with a hope of working on your system, then... You still need to copy the files from the distribution directory "python/Lib" to your system. If you don't have the full distribution, you can get the file lib<version>.tar.gz from most ftp sites carrying Python; this is a subset of the distribution containing just those files, e.g. ftp://ftp.python.org/pub/python/src/lib1.4.tar.gz. Once you have installed the library, you need to point sys.path to it. Assuming the library is in C:\misc\python\lib, the following commands will point your Python interpreter to it (note the doubled backslashes -- you can also use single forward slashes instead): >>> import sys >>> sys.path.insert(0, 'C:\\misc\\python\\lib') >>> For a more permanent effect, set the environment variable PYTHONPATH, as follows (talking to a DOS prompt): C> SET PYTHONPATH=C:\misc\python\lib ---------------------------------------------------------------------------- 7.9. Where's the documentation for the Mac or PC version? The documentation for the Unix version also applies to the Mac and PC versions. Where applicable, differences are indicated in the text. ---------------------------------------------------------------------------- 7.10. How do I create a Python program file on the Mac or PC? Use an external editor. On the Mac, BBEdit seems to be a popular no-frills text editor. I work like this: start the interpreter; edit a module file using BBedit; import and test it in the interpreter; edit again in BBedit; then use the built-in function reload() to re-read the imported module; etc. In the 1.4 distribution you will find a BBEdit extension that makes life a little easier: it can tell the interpreter to execute the current window. See :Mac:Tools:BBPy:README. Regarding the same question for the PC, Kurt Wm. Hemr writes: "While anyone with a pulse could certainly figure out how to do the same on MS-Windows, I would recommend the NotGNU Emacs clone for MS-Windows. Not only can you easily resave and "reload()" from Python after making changes, but since WinNot auto-copies to the clipboard any text you select, you can simply select the entire procedure (function) which you changed in WinNot, switch to QWPython, and shift-ins to reenter the changed program unit." If you're using Windows95 or Windows NT, you should also know about PythonWin, which provides a GUI framework, with an mouse-driven editor, an object browser, and a GUI-based debugger. See http://www.python.org/ftp/python/pythonwin/ for details. ---------------------------------------------------------------------------- 7.11. How can I use Tkinter on Windows 95/NT? Starting from Python 1.5, it's very easy -- just download and install Python and Tcl/Tk and you're in business. See http://www.python.org/download/download_windows.html One warning: don't attempt to use Tkinter from PythonWin (Mark Hammond's IDE). Use it from the command line interface (python.exe) or the windowless interpreter (pythonw.exe). ---------------------------------------------------------------------------- 7.12. cgi.py (or other CGI programming) doesn't work sometimes on NT or win95! Be sure you have the latest python.exe, that you are using python.exe rather than a GUI version of python and that you have configured the server to execute "...\python.exe -u ..." for the cgi execution. The -u (unbuffered) option on NT and win95 prevents the interpreter from altering newlines in the standard input and output. Without it post/multipart requests will seem to have the wrong length and binary (eg, GIF) responses may get garbled (resulting in, eg, a "broken image"). ---------------------------------------------------------------------------- 7.13. Why doesn't os.popen() work in PythonWin on NT? The reason that os.popen() doesn't work from within PythonWin is due to a bug in Microsoft's C Runtime Library (CRT). The CRT assumes you have a Win32 console attached to the process. You should use the win32pipe module's popen() instead which doesn't depend on having an attached Win32 console. Example: import win32pipe f = win32pipe.popen('dir /c c:\\') print f.readlines() f.close() ---------------------------------------------------------------------------- 7.14. How do I use different functionality on different platforms with the same program? Remember that Python is extremely dynamic and that you can use this dynamism to configure a program at run-time to use available functionality on different platforms. For example you can test the sys.platform and import different modules based on its value. import sys if sys.platform == "win32": import win32pipe popen = win32pipe.popen else: import os popen = os.popen (See FAQ 7.13 for an explanation of why you might want to do something like this.) Also you can try to import a module and use a fallback if the import fails: try: import really_fast_implementation choice = really_fast_implementation except ImportError: import slower_implementation choice = slower_implementation ---------------------------------------------------------------------------- 7.15. Is there an Amiga version of Python? Yes. See the AmigaPython homepage at http://www.bigfoot.com/~irmen/python.html. ---------------------------------------------------------------------------- 7.16. Why doesn't os.popen()/win32pipe.popen() work on Win9x? There is a bug in Win9x that prevents os.popen/win32pipe.popen* from working. The good news is there is a way to work around this problem. The Microsoft Knowledge Base article that you need to lookup is: Q150956. You will find links to the knowledge base at: http://www.microsoft.com/kb. ---------------------------------------------------------------------------- -- ----------- comp.lang.python.announce (moderated) ---------- Article Submission Address: python-announce(a)python.org Python Language Home Page: http://www.python.org/ Python Quick Help Index: http://www.python.org/Help.html ------------------------------------------------------------

1 0

Python Language FAQ - Section 6
by Markus Fleck 07 Jul '99

07 Jul '99

This FAQ newsgroup posting has been automatically converted from an HTML snapshot of the original Python FAQ; please refer to the original "Python FAQ Wizard" at <http://grail.cnri.reston.va.us/cgi-bin/faqw.py> if source code snippets given in this document do not work - incidentally some formatting information may have been lost during the conversion. ---------------------------------------------------------------------------- The whole Python FAQ - Section 6 Last changed on Mon Jun 28 19:36:09 1999 EDT (Entries marked with ** were changed within the last 24 hours; entries marked with * were changed within the last 7 days.) ---------------------------------------------------------------------------- 6. Python's design 6.1. Why isn't there a switch or case statement in Python? 6.2. Why does Python use indentation for grouping of statements? 6.3. Why are Python strings immutable? 6.4. Why don't strings have methods like index() or sort(), like lists? 6.5. Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))? 6.6. Why can't I derive a class from built-in types (e.g. lists or files)? 6.7. Why must 'self' be declared and used explicitly in method definitions and calls? 6.8. Can't you emulate threads in the interpreter instead of relying on an OS-specific thread implementation? 6.9. Why can't lambda forms contain statements? 6.10. Why don't lambdas have access to variables defined in the containing scope? 6.11. Why can't recursive functions be defined inside other functions? 6.12. Why is there no more efficient way of iterating over a dictionary than first constructing the list of keys()? 6.13. Can Python be compiled to machine code, C or some other language? 6.14. How does Python manage memory? Why not full garbage collection? 6.15. Why are there separate tuple and list data types? 6.16. How are lists implemented? 6.17. How are dictionaries implemented? 6.18. Why must dictionary keys be immutable? 6.19. How the heck do you make an array in Python? 6.20. Why doesn't list.sort() return the sorted list? 6.21. How do you specify and enforce an interface spec in Python? 6.22. Why do all classes have the same type? Why do instances all have the same type? 6.23. Why isn't all memory freed when Python exits? 6.24. Why no class methods or mutable class variables? 6.25. Why are default values sometimes shared between objects? 6.26. Why no goto? 6.27. How do you make a higher order function in Python? 6.28. Why do I get a SyntaxError for a 'continue' inside a 'try'? 6.29. Why can't raw strings (r-strings) end with a backslash? 6.30. Why can't I use an assignment in an expression? ---------------------------------------------------------------------------- 6. Python's design ---------------------------------------------------------------------------- 6.1. Why isn't there a switch or case statement in Python? You can do this easily enough with a sequence of if... elif... elif... else. There have been some proposals for switch statement syntax, but there is no consensus (yet) on whether and how to do range tests. ---------------------------------------------------------------------------- 6.2. Why does Python use indentation for grouping of statements? Basically I believe that using indentation for grouping is extremely elegant and contributes a lot to the clarity of the average Python program. Most people learn to love this feature after a while. Some arguments for it: Since there are no begin/end brackets there cannot be a disagreement between grouping perceived by the parser and the human reader. I remember long ago seeing a C fragment like this: if (x <= y) x++; y--; z++; and staring a long time at it wondering why y was being decremented even for x > y... (And I wasn't a C newbie then either.) Since there are no begin/end brackets, Python is much less prone to coding-style conflicts. In C there are loads of different ways to place the braces (including the choice whether to place braces around single statements in certain cases, for consistency). If you're used to reading (and writing) code that uses one style, you will feel at least slightly uneasy when reading (or being required to write) another style. Many coding styles place begin/end brackets on a line by themself. This makes programs considerably longer and wastes valuable screen space, making it harder to get a good overview over a program. Ideally, a function should fit on one basic tty screen (say, 20 lines). 20 lines of Python are worth a LOT more than 20 lines of C. This is not solely due to the lack of begin/end brackets (the lack of declarations also helps, and the powerful operations of course), but it certainly helps! ---------------------------------------------------------------------------- 6.3. Why are Python strings immutable? There are two advantages. One is performance: knowing that a string is immutable makes it easy to lay it out at construction time -- fixed and unchanging storage requirements. (This is also one of the reasons for the distinction between tuples and lists.) The other is that strings in Python are considered as "elemental" as numbers. No amount of activity will change the value 8 to anything else, and in Python, no amount of activity will change the string "eight" to anything else. (Adapted from Jim Roskind) ---------------------------------------------------------------------------- 6.4. Why don't strings have methods like index() or sort(), like lists? Good question. Strings currently don't have methods at all (likewise tuples and numbers). Long ago, it seemed unnecessary to implement any of these functions in C, so a standard library module "string" written in Python was created that performs string related operations. Since then, the cry for performance has moved most of them into the built-in module strop (this is imported by module string, which is still the preferred interface, without loss of performance except during initialization). Some of these functions (e.g. index()) could easily be implemented as string methods instead, but others (e.g. sort()) can't, since their interface prescribes that they modify the object, while strings are immutable (see the previous question). ---------------------------------------------------------------------------- 6.5. Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))? Functions are used for those operations that are generic for a group of types and which should work even for objects that don't have methods at all (e.g. numbers, strings, tuples). Also, implementing len(), max(), min() as a built-in function is actually less code than implementing them as methods for each type. One can quibble about individual cases but it's really too late to change such things fundamentally now. ---------------------------------------------------------------------------- 6.6. Why can't I derive a class from built-in types (e.g. lists or files)? This is caused by the relatively late addition of (user-defined) classes to the language -- the implementation framework doesn't easily allow it. See the answer to question 4.2 for a work-around. This may be fixed in the (distant) future. ---------------------------------------------------------------------------- 6.7. Why must 'self' be declared and used explicitly in method definitions and calls? By asking this question you reveal your C++ background. :-) When I added classes, this was (again) the simplest way of implementing methods without too many changes to the interpreter. I borrowed the idea from Modula-3. It turns out to be very useful, for a variety of reasons. First, it makes it more obvious that you are using a method or instance attribute instead of a local variable. Reading "self.x" or "self.meth()" makes it absolutely clear that an instance variable or method is used even if you don't know the class definition by heart. In C++, you can sort of tell by the lack of a local variable declaration (assuming globals are rare or easily recognizable) -- but in Python, there are no local variable declarations, so you'd have to look up the class definition to be sure. Second, it means that no special syntax is necessary if you want to explicitly reference or call the method from a particular class. In C++, if you want to use a method from base class that is overridden in a derived class, you have to use the :: operator -- in Python you can write baseclass.methodname(self, <argument list>). This is particularly useful for __init__() methods, and in general in cases where a derived class method wants to extend the base class method of the same name and thus has to call the base class method somehow. Lastly, for instance variables, it solves a syntactic problem with assignment: since local variables in Python are (by definition!) those variables to which a value assigned in a function body (and that aren't explicitly declared global), there has to be some way to tell the interpreter that an assignment was meant to assign to an instance variable instead of to a local variable, and it should preferably be syntactic (for efficiency reasons). C++ does this through declarations, but Python doesn't have declarations and it would be a pity having to introduce them just for this purpose. Using the explicit "self.var" solves this nicely. Similarly, for using instance variables, having to write "self.var" means that references to unqualified names inside a method don't have to search the instance's directories. ---------------------------------------------------------------------------- 6.8. Can't you emulate threads in the interpreter instead of relying on an OS-specific thread implementation? Unfortunately, the interpreter pushes at least one C stack frame for each Python stack frame. Also, extensions can call back into Python at almost random moments. Therefore a complete threads implementation requires thread support for C. ---------------------------------------------------------------------------- 6.9. Why can't lambda forms contain statements? Python lambda forms cannot contain statements because Python's syntactic framework can't handle statements nested inside expressions. However, in Python, this is not a serious problem. Unlike lambda forms in other languages, where they add functionality, Python lambdas are only a shorthand notation if you're too lazy to define a function. Functions are already first class objects in Python, and can be declared in a local scope. Therefore the only advantage of using a lambda form instead of a locally-defined function is that you don't need to invent a name for the function -- but that's just a local variable to which the function object (which is exactly the same type of object that a lambda form yields) is assigned! ---------------------------------------------------------------------------- 6.10. Why don't lambdas have access to variables defined in the containing scope? Because they are implemented as ordinary functions. See question 4.5 above. ---------------------------------------------------------------------------- 6.11. Why can't recursive functions be defined inside other functions? See question 4.5 above. But actually recursive functions can be defined in other functions with some trickery. def test(): class factorial: def __call__(self, n): if n<=1: return 1 return n * self(n-1) return factorial() fact = test() The instance created by factorial() above acts like the recursive factorial function. Mutually recursive functions can be passed to each other as arguments. ---------------------------------------------------------------------------- 6.12. Why is there no more efficient way of iterating over a dictionary than first constructing the list of keys()? Have you tried it? I bet it's fast enough for your purposes! In most cases such a list takes only a few percent of the space occupied by the dictionary. Apart from the fixed header, the list needs only 4 bytes (the size of a pointer) per key. A dictionary uses 12 bytes per key plus between 30 and 70 percent hash table overhead, plus the space for the keys and values. By necessity, all keys are distinct objects, and a string object (the most common key type) costs at least 20 bytes plus the length of the string. Add to that the values contained in the dictionary, and you see that 4 bytes more per item really isn't that much more memory... A call to dict.keys() makes one fast scan over the dictionary (internally, the iteration function does exist) copying the pointers to the key objects into a pre-allocated list object of the right size. The iteration time isn't lost (since you'll have to iterate anyway -- unless in the majority of cases your loop terminates very prematurely (which I doubt since you're getting the keys in random order). I don't expose the dictionary iteration operation to Python programmers because the dictionary shouldn't be modified during the entire iteration -- if it is, there's a small chance that the dictionary is reorganized because the hash table becomes too full, and then the iteration may miss some items and see others twice. Exactly because this only occurs rarely, it would lead to hidden bugs in programs: it's easy never to have it happen during test runs if you only insert or delete a few items per iteration -- but your users will surely hit upon it sooner or later. ---------------------------------------------------------------------------- 6.13. Can Python be compiled to machine code, C or some other language? Not easily. Python's high level data types, dynamic typing of objects and run-time invocation of the interpreter (using eval() or exec) together mean that a "compiled" Python program would probably consist mostly of calls into the Python run-time system, even for seemingly simple operations like "x+1". Thus, the performance gain would probably be minimal. Internally, Python source code is always translated into a "virtual machine code" or "byte code" representation before it is interpreted (by the "Python virtual machine" or "bytecode interpreter"). In order to avoid the overhead of parsing and translating modules that rarely change over and over again, this byte code is written on a file whose name ends in ".pyc" whenever a module is parsed (from a file whose name ends in ".py"). When the corresponding .py file is changed, it is parsed and translated again and the .pyc file is rewritten. There is no performance difference once the .pyc file has been loaded (the bytecode read from the .pyc file is exactly the same as the bytecode created by direct translation). The only difference is that loading code from a .pyc file is faster than parsing and translating a .py file, so the presence of precompiled .pyc files will generally improve start-up time of Python scripts. If desired, the Lib/compileall.py module/script can be used to force creation of valid .pyc files for a given set of modules. Note that the main script executed by Python, even if its filename ends in .py, is not compiled to a .pyc file. It is compiled to bytecode, but the bytecode is not saved to a file. If you are looking for a way to translate Python programs in order to distribute them in binary form, without the need to distribute the interpreter and library as well, have a look at the freeze.py script in the Tools/freeze directory. This creates a single binary file incorporating your program, the Python interpreter, and those parts of the Python library that are needed by your program. Of course, the resulting binary will only run on the same type of platform as that used to create it. ---------------------------------------------------------------------------- 6.14. How does Python manage memory? Why not full garbage collection? The details of Python memory management depend on the implementation. The standard Python implementation (the C implementation) uses reference counting memory management. This means that when an object is no longer in use Python frees the object automatically, with a few exceptions. On the other hand, JPython relies on the Java runtime; so it uses the JVM's garbage collector. This difference can cause some subtle porting problems if your Python code depends on the behavior of the reference counting implementation. Two exceptions to bear in mind for standard Python are: 1) if the object lies on a circular reference path it won't be freed unless the circularities are broken. EG: List = [None] List[0] = List List will not be freed unless the circularity (List[0] is List) is broken. The reason List will not be freed is because although it may become inaccessible the list contains a reference to itself, and reference counting only deallocates an object when all references to an object are destroyed. To break the circular reference path we must destroy the reference, as in List[0] = None So, if your program creates circular references (and if it is long running and/or consumes lots of memory) it may have to do some explicit management of circular structures. In many application domains this is needed rarely, if ever. 2) Sometimes objects get stuck in "tracebacks" temporarily and hence are not deallocated when you might expect. Clear the tracebacks via import sys sys.exc_traceback = sys.last_traceback = None Tracebacks are used for reporting errors and implementing debuggers and related things. They contain a portion of the program state extracted during the handling of an exception (usually the most recent exception). In the absence of circularities and modulo tracebacks, Python programs need not explicitly manage memory. It is often suggested that Python could benefit from fully general garbage collection. It's looking less and less likely that Python will ever get "automatic" garbage collection (GC). For one thing, unless this were added to C as a standard feature, it's a portability pain in the ass. And yes, I know about the Xerox library. It has bits of assembler code for most common platforms. Not for all. And although it is mostly transparent, it isn't completely transparent (when I once linked Python with it, it dumped core). "Proper" GC also becomes a problem when Python gets embedded into other applications. While in a stand-alone Python it may be fine to replace the standard malloc() and free() with versions provided by the GC library, an application embedding Python may want to have its own substitute for malloc() and free(), and may not want Python's. Right now, Python works with anything that implements malloc() and free() properly. In JPython, which has garbage collection, the following code (which is fine in C Python) will probably run out of file descriptors long before it runs out of memory: for file in <very long list of files>: f = open(file) c = f.read(1) Using the current reference counting and destructor scheme, each new assignment to f closes the previous file. Using GC, this is not guaranteed. Sure, you can think of ways to fix this. But it's not off-the-shelf technology. If you want to write code that will work with any Python implementation, you should explicitly close the file; this will work regardless of GC: for file in <very long list of files>: f = open(file) c = f.read(1) f.close() All that said, somebody has managed to add GC to Python using the GC library fromn Xerox, so you can see for yourself. See http://starship.python.net/crew/gandalf/gc-ss.html See also question 4.17 for ways to plug some common memory leaks manually. If you're not satisfied with the answers here, before you post to the newsgroup, please read this summary of past discussions on GC for Python by Moshe Zadka: http://www.geocities.com/TheTropics/Island/2932/gcpy.html ---------------------------------------------------------------------------- 6.15. Why are there separate tuple and list data types? This is done so that tuples can be immutable while lists are mutable. Immutable tuples are useful in situations where you need to pass a few items to a function and don't want the function to modify the tuple; for example, point1 = (120, 140) point2 = (200, 300) record(point1, point2) draw(point1, point2) You don't want to have to think about what would happen if record() changed the coordinates -- it can't, because the tuples are immutable. On the other hand, when creating large lists dynamically, it is absolutely crucial that they are mutable -- adding elements to a tuple one by one requires using the concatenation operator, which makes it quadratic in time. As a general guideline, use tuples like you would use structs in C or records in Pascal, use lists like (variable length) arrays. ---------------------------------------------------------------------------- 6.16. How are lists implemented? Despite what a Lisper might think, Python's lists are really variable-length arrays. The implementation uses a contiguous array of references to other objects, and keeps a pointer to this array (as well as its length) in a list head structure. This makes indexing a list (a[i]) an operation whose cost is independent of the size of the list or the value of the index. When items are appended or inserted, the array of references is resized. Some cleverness is applied to improve the performance of appending items repeatedly; when the array must be grown, some extra space is allocated so the next few times don't require an actual resize. ---------------------------------------------------------------------------- 6.17. How are dictionaries implemented? Python's dictionaries are implemented as resizable hash tables. Compared to B-trees, this gives better performance for lookup (the most common operation by far) under most circumstances, and the implementation is simpler. ---------------------------------------------------------------------------- 6.18. Why must dictionary keys be immutable? The hash table implementation of dictionaries uses a hash value calculated from the key value to find the key. If the key were a mutable object, its value could change, and thus its hash could change. But since whoever changes the key object can't tell that is incorporated in a dictionary, it can't move the entry around in the dictionary. Then, when you try to look up the same object in the dictionary, it won't be found, since its hash value is different; and if you try to look up the old value, it won't be found either, since the value of the object found in that hash bin differs. If you think you need to have a dictionary indexed with a list, try to use a tuple instead. The function tuple(l) creates a tuple with the same entries as the list l. Some unacceptable solutions that have been proposed: - Hash lists by their address (object ID). This doesn't work because if you construct a new list with the same value it won't be found; e.g., d = {[1,2]: '12'} print d[[1,2]] will raise a KeyError exception because the id of the [1,2] used in the second line differs from that in the first line. In other words, dictionary keys should be compared using '==', not using 'is'. - Make a copy when using a list as a key. This doesn't work because the list (being a mutable object) could contain a reference to itself, and then the copying code would run into an infinite loop. - Allow lists as keys but tell the user not to modify them. This would allow a class of hard-to-track bugs in programs that I'd rather not see; it invalidates an important invariant of dictionaries (every value in d.keys() is usable as a key of the dictionary). - Mark lists as read-only once they are used as a dictionary key. The problem is that it's not just the top-level object that could change its value; you could use a tuple containing a list as a key. Entering anything as a key into a dictionary would require marking all objects reachable from there as read-only -- and again, self-referential objects could cause an infinite loop again (and again and again). There is a trick to get around this if you need to, but use it at your own risk: You can wrap a mutable structure inside a class instance which has both a __cmp__ and a __hash__ method. class listwrapper: def __init__(self, the_list): self.the_list = the_list def __cmp__(self, other): return self.the_list == other.the_list def __hash__(self): l = self.the_list result = 98767 - len(l)*555 for i in range(len(l)): try: result = result + (hash(l[i]) % 9999999) * 1001 + i except: result = (result % 7777777) + i * 333 return result Note that the hash computation is complicated by the possibility that some members of the list may be unhashable and also by the possibility of arithmetic overflow. You must make sure that the hash value for all such wrapper objects that reside in a dictionary (or other hash based structure), remain fixed while the object is in the dictionary (or other structure). Furthermore it must always be the case that if o1 == o2 (ie o1.__cmp__(o2)==0) then hash(o1)==hash(o2) (ie, o1.__hash__() == o2.__hash__()), regardless of whether the object is in a dictionary or not. If you fail to meet these restrictions dictionaries and other hash based structures may misbehave! In the case of listwrapper above whenever the wrapper object is in a dictionary the wrapped list must not change to avoid anomalies. Don't do this unless you are prepared to think hard about the requirements and the consequences of not meeting them correctly. You've been warned! ---------------------------------------------------------------------------- 6.19. How the heck do you make an array in Python? ["this", 1, "is", "an", "array"] Lists are arrays in the C or Pascal sense of the word (see question 6.16). The array module also provides methods for creating arrays of fixed types with compact representations (but they are slower to index than lists). Also note that the Numerics extensions and others define array-like structures with various characteristics as well. To get Lisp-like lists, emulate cons cells lisp_list = ("like", ("this", ("example", None) ) ) using tuples (or lists, if you want mutability). Here the analogue of lisp car is lisp_list[0] and the analogue of cdr is lisp_list[1]. Only do this if you're sure you really need to (it's usually a lot slower than using Python lists). Think of Python lists as mutable heterogeneous arrays of Python objects (say that 10 times fast :) ). ---------------------------------------------------------------------------- 6.20. Why doesn't list.sort() return the sorted list? In situations where performance matters, making a copy of the list just to sort it would be wasteful. Therefore, list.sort() sorts the list in place. In order to remind you of that fact, it does not return the sorted list. This way, you won't be fooled into accidentally overwriting a list when you need a sorted copy but also need to keep the unsorted version around. As a result, here's the idiom to iterate over the keys of a dictionary in sorted orted: keys = dict.keys() keys.sort() for key in keys: ...do whatever with dict[key]... ---------------------------------------------------------------------------- 6.21. How do you specify and enforce an interface spec in Python? An interfaces specification for a module as provided by languages such as C++ and java describes the prototypes for the methods and functions of the module. Many feel that compile time enforcement of interface specifications help aid in the construction of large programs. Python does not support interface specifications directly, but many of their advantages can be obtained by an appropriate test discipline for components, which can often be very easily accomplished in Python. A good test suite for a module can at once provide a regression test and serve as a module interface specification (even better since it also gives example usage). Look to many of the standard libraries which often have a "script interpretation" which provides a simple "self test." Even modules which use complex external interfaces can often be tested in isolation using trivial "stub" emulations of the external interface. An appropriate testing discipline (if enforced) can help build large complex applications in Python as well as having interface specifications would do (or better). Of course Python allows you to get sloppy and not do it. Also you might want to design your code with an eye to make it easily tested. ---------------------------------------------------------------------------- 6.22. Why do all classes have the same type? Why do instances all have the same type? The Pythonic use of the word "type" is quite different from common usage in much of the rest of the programming language world. A "type" in Python is a description for an object's operations as implemented in C. All classes have the same operations implemented in C which sometimes "call back" to differing program fragments implemented in Python, and hence all classes have the same type. Similarly at the C level all class instances have the same C implementation, and hence all instances have the same type. Remember that in Python usage "type" refers to a C implementation of an object. To distinguish among instances of different classes use Instance.__class__, and also look to 4.47. Sorry for the terminological confusion, but at this point in Python's development nothing can be done! ---------------------------------------------------------------------------- 6.23. Why isn't all memory freed when Python exits? Objects referenced from Python module global name spaces are not always deallocated when Python exits. This may happen if there are circular references (see question 4.17). There are also certain bits of memory that are allocated by the C library that are impossible to free (e.g. a tool like Purify will complain about these). But in general, Python 1.5 and beyond (in contrast with earlier versions) is quite agressive about cleaning up memory on exit. If you want to force Python to delete certain things on deallocation use the sys.exitfunc hook to force those deletions. For example if you are debugging an extension module using a memory analysis tool and you wish to make Python deallocate almost everything you might use an exitfunc like this one: import sys def my_exitfunc(): print "cleaning up" import sys # do order dependant deletions here ... # now delete everything else in arbitrary order for x in sys.modules.values(): d = x.__dict__ for name in d.keys(): del d[name] sys.exitfunc = my_exitfunc Other exitfuncs can be less drastic, of course. (In fact, this one just does what Python now already does itself; but the example of using sys.exitfunc to force cleanups is still useful.) ---------------------------------------------------------------------------- 6.24. Why no class methods or mutable class variables? The notation instance.attribute(arg1, arg2) usually translates to the equivalent of Class.attribute(instance, arg1, arg2) where Class is a (super)class of instance. Similarly instance.attribute = value sets an attribute of an instance (overriding any attribute of a class that instance inherits). Sometimes programmers want to have different behaviours -- they want a method which does not bind to the instance and a class attribute which changes in place. Python does not preclude these behaviours, but you have to adopt a convention to implement them. One way to accomplish this is to use "list wrappers" and global functions. def C_hello(): print "hello" class C: hello = [C_hello] counter = [0] I = C() Here I.hello[0]() acts very much like a "class method" and I.counter[0] = 2 alters C.counter (and doesn't override it). If you don't understand why you'd ever want to do this, that's because you are pure of mind, and you probably never will want to do it! This is dangerous trickery, not recommended when avoidable. (Inspired by Tim Peter's discussion.) ---------------------------------------------------------------------------- 6.25. Why are default values sometimes shared between objects? It is often expected that a function CALL creates new objects for default values. This is not what happens. Default values are created when the function is DEFINED, that is, there is only one such object that all functions refer to. If that object is changed, subsequent calls to the function will refer to this changed object. By definition, immutable objects (like numbers, strings, tuples, None) are safe from change. Changes to mutable objects (like dictionaries, lists, class instances) is what causes the confusion. Because of this feature it is good programming practice not to use mutable objects as default values, but to introduce them in the function. Don't write: def foo(dict={}): # XXX shared reference to one dict for all calls ... but: def foo(dict=None): if dict is None: dict = {} # create a new dict for local namespace See page 182 of "Internet Programming with Python" for one discussion of this feature. Or see the top of page 144 or bottom of page 277 in "Programming Python" for another discussion. ---------------------------------------------------------------------------- 6.26. Why no goto? Actually, you can use exceptions to provide a "structured goto" that even works across function calls. Many feel that exceptions can conveniently emulate all reasonable uses of the "go" or "goto" constructs of C, Fortran, and other languages. For example: class label: pass # declare a label try: ... if (condition): raise label() # goto label ... except label: # where to goto pass ... This doesn't allow you to jump into the middle of a loop, but that's usually considered an abuse of goto anyway. Use sparingly. ---------------------------------------------------------------------------- 6.27. How do you make a higher order function in Python? You have two choices: you can use default arguments and override them or you can use "callable objects." For example suppose you wanted to define linear(a,b) which returns a function f where f(x) computes the value a*x+b. Using default arguments: def linear(a,b): def result(x, a=a, b=b): return a*x + b return result Or using callable objects: class linear: def __init__(self, a, b): self.a, self.b = a,b def __call__(self, x): return self.a * x + self.b In both cases: taxes = linear(0.3,2) gives a callable object where taxes(10e6) == 0.3 * 10e6 + 2. The defaults strategy has the disadvantage that the default arguments could be accidentally or maliciously overridden. The callable objects approach has the disadvantage that it is a bit slower and a bit longer. Note however that a collection of callables can share their signature via inheritance. EG class exponential(linear): # __init__ inherited def __call__(self, x): return self.a * (x ** self.b) On comp.lang.python, zenin(a)bawdycaste.org points out that an object can encapsulate state for several methods in order to emulate the "closure" concept from functional programming languages, for example: class counter: value = 0 def set(self, x): self.value = x def up(self): self.value=self.value+1 def down(self): self.value=self.value-1 count = counter() inc, dec, reset = count.up, count.down, count.set Here inc, dec and reset act like "functions which share the same closure containing the variable count.value" (if you like that way of thinking). ---------------------------------------------------------------------------- 6.28. Why do I get a SyntaxError for a 'continue' inside a 'try'? This is an implementation limitation, caused by the extremely simple-minded way Python generates bytecode. The try block pushes something on the "block stack" which the continue would have to pop off again. The current code generator doesn't have the data structures around so that 'continue' can generate the right code. Note that JPython doesn't have this restriction! ---------------------------------------------------------------------------- 6.29. Why can't raw strings (r-strings) end with a backslash? More precisely, they can't end with an odd number of backslashes: the unpaired backslash at the end escapes the closing quote character, leaving an unterminated string. Raw strings were designed to ease creating input for processors (chiefly regular expression engines) that want to do their own backslash escape processing. Such processors consider an unmatched trailing backslash to be an error anyway, so raw strings disallow that. In return, they allow you to pass on the string quote character by escaping it with a backslash. These rules work well when r-strings are used for their intended purpose. If you're trying to build Windows pathnames, note that all Windows system calls accept forward slashes too: f = open("/mydir/file.txt") # works fine! If you're trying to build a pathname for a DOS command, try e.g. one of dir = r"\this\is\my\dos\dir" "\\" dir = r"\this\is\my\dos\dir\ "[:-1] dir = "\\this\\is\\my\\dos\\dir\\" ---------------------------------------------------------------------------- 6.30. Why can't I use an assignment in an expression? Many people used to C or Perl complain that they want to be able to use e.g. this C idiom: while (line = readline(f)) { ...do something with line... } where in Python you're forced to write this: while 1: line = f.readline() if not line: break ...do something with line... This issue comes up in the Python newsgroup with alarming frequency -- search Deja News for past messages about assignment expression. The reason for not allowing assignment in Python expressions is a common, hard-to-find bug in those other languages, caused by this construct: if (x = 0) { ...error handling... } else { ...code that only works for nonzero x... } Many alternatives have been proposed. Most are hacks that save some typing but use arbitrary or cryptic syntax or keywords, and fail the simple criterion that I use for language change proposals: it should intuitively suggest the proper meaning to a human reader who has not yet been introduced with the construct. The earliest time something can be done about this will be with Python 2.0 -- if it is decided that it is worth fixing. An interesting phenomenon is that most experienced Python programmers recognize the "while 1" idiom and don't seem to be missing the assignment in expression construct much; it's only the newcomers who express a strong desire to add this to the language. One fairly elegant solution would be to introduce a new operator for assignment in expressions spelled ":=" -- this avoids the "=" instead of "==" problem. It would have the same precedence as comparison operators but the parser would flag combination with other comparisons (without disambiguating parentheses) as an error. Finally -- there's an alternative way of spelling this that seems attractive but is generally less robust than the "while 1" solution: line = f.readline() while line: ...do something with line... line = f.readline() The problem with this is that if you change your mind about exactly how you get the next line (e.g. you want to change it into sys.stdin.readline()) you have to remember to change two places in your program -- the second one hidden at the bottom of the loop. ---------------------------------------------------------------------------- -- ----------- comp.lang.python.announce (moderated) ---------- Article Submission Address: python-announce(a)python.org Python Language Home Page: http://www.python.org/ Python Quick Help Index: http://www.python.org/Help.html ------------------------------------------------------------

1 0

Python Language FAQ - Section 5
by Markus Fleck 07 Jul '99

07 Jul '99

This FAQ newsgroup posting has been automatically converted from an HTML snapshot of the original Python FAQ; please refer to the original "Python FAQ Wizard" at <http://grail.cnri.reston.va.us/cgi-bin/faqw.py> if source code snippets given in this document do not work - incidentally some formatting information may have been lost during the conversion. ---------------------------------------------------------------------------- The whole Python FAQ - Section 5 Last changed on Mon Jun 28 19:36:09 1999 EDT (Entries marked with ** were changed within the last 24 hours; entries marked with * were changed within the last 7 days.) ---------------------------------------------------------------------------- 5. Extending Python 5.1. Can I create my own functions in C? 5.2. Can I create my own functions in C++? 5.3. How can I execute arbitrary Python statements from C? 5.4. How can I evaluate an arbitrary Python expression from C? 5.5. How do I extract C values from a Python object? 5.6. How do I use Py_BuildValue() to create a tuple of arbitrary length? 5.7. How do I call an object's method from C? 5.8. How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)? 5.9. How do I access a module written in Python from C? 5.10. How do I interface to C++ objects from Python? 5.11. mSQLmodule (or other old module) won't build with Python 1.5 (or later) 5.12. I added a module using the Setup file and the make fails! Huh? 5.13. I want to compile a Python module on my Red Hat Linux system, but some files are missing. 5.14. What does "SystemError: _PyImport_FixupExtension: module yourmodule not loaded" mean? ---------------------------------------------------------------------------- 5. Extending Python ---------------------------------------------------------------------------- 5.1. Can I create my own functions in C? Yes, you can create built-in modules containing functions, variables, exceptions and even new types in C. This is explained in the document "Extending and Embedding the Python Interpreter" (the LaTeX file Doc/ext.tex). Also read the chapter on dynamic loading. There's more information on this in each of the Python books: Programming Python, Internet Programming with Python, and Das Python-Buch (in German). ---------------------------------------------------------------------------- 5.2. Can I create my own functions in C++? Yes, using the C-compatibility features found in C++. Basically you place extern "C" { ... } around the Python include files and put extern "C" before each function that is going to be called by the Python interpreter. Global or static C++ objects with constructors are probably not a good idea. ---------------------------------------------------------------------------- 5.3. How can I execute arbitrary Python statements from C? The highest-level function to do this is PyRun_SimpleString() which takes a single string argument which is executed in the context of module __main__ and returns 0 for success and -1 when an exception occurred (including SyntaxError). If you want more control, use PyRun_String(); see the source for PyRun_SimpleString() in Python/pythonrun.c. ---------------------------------------------------------------------------- 5.4. How can I evaluate an arbitrary Python expression from C? Call the function PyRun_String() from the previous question with the start symbol eval_input (Py_eval_input starting with 1.5a1); it parses an expression, evaluates it and returns its value. ---------------------------------------------------------------------------- 5.5. How do I extract C values from a Python object? That depends on the object's type. If it's a tuple, PyTupleSize(o) returns its length and PyTuple_GetItem(o, i) returns its i'th item; similar for lists with PyListSize(o) and PyList_GetItem(o, i). For strings, PyString_Size(o) returns its length and PyString_AsString(o) a pointer to its value (note that Python strings may contain null bytes so strlen() is not safe). To test which type an object is, first make sure it isn't NULL, and then use PyString_Check(o), PyTuple_Check(o), PyList_Check(o), etc. There is also a high-level API to Python objects which is provided by the so-called 'abstract' interface -- read Include/abstract.h for further details. It allows for example interfacing with any kind of Python sequence (e.g. lists and tuples) using calls like PySequence_Length(), PySequence_GetItem(), etc.) as well as many other useful protocols. ---------------------------------------------------------------------------- 5.6. How do I use Py_BuildValue() to create a tuple of arbitrary length? You can't. Use t = PyTuple_New(n) instead, and fill it with objects using PyTuple_SetItem(t, i, o) -- note that this "eats" a reference count of o. Similar for lists with PyList_New(n) and PyList_SetItem(l, i, o). Note that you must set all the tuple items to some value before you pass the tuple to Python code -- PyTuple_New(n) initializes them to NULL, which isn't a valid Python value. ---------------------------------------------------------------------------- 5.7. How do I call an object's method from C? Here's a function (untested) that might become part of the next release in some form. It uses <stdarg.h> to allow passing the argument list on to vmkvalue(): object *call_method(object *inst, char *methodname, char *format, ...) { object *method; object *args; object *result; va_list va; method = getattr(inst, methodname); if (method == NULL) return NULL; va_start(va, format); args = vmkvalue(format, va); va_end(va); if (args == NULL) { DECREF(method); return NULL; } result = call_object(method, args); DECREF(method); DECREF(args); return result; } This works for any instance that has methods -- whether built-in or user-defined. You are responsible for eventually DECREF'ing the return value. To call, e.g., a file object's "seek" method with arguments 10, 0 (assuming the file object pointer is "f"): res = call_method(f, "seek", "(OO)", 10, 0); if (res == NULL) { ... an exception occurred ... } else { DECREF(res); } Note that since call_object() always wants a tuple for the argument list, to call a function without arguments, pass "()" for the format, and to call a function with one argument, surround the argument in parentheses, e.g. "(i)". ---------------------------------------------------------------------------- 5.8. How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)? (Due to Mark Hammond): In Python code, define an object that supports the "write()" method. Redirect sys.stdout and sys.stderr to this object. Call print_error, or just allow the standard traceback mechanism to work. Then, the output will go wherever your write() method sends it. The easiest way to do this is to use the StringIO class in the standard library. Sample code and use for catching stdout: >>> class StdoutCatcher: ... def __init__(self): ... self.data = '' ... def write(self, stuff): ... self.data = self.data + stuff ... >>> import sys >>> sys.stdout = StdoutCatcher() >>> print 'foo' >>> print 'hello world!' >>> sys.stderr.write(sys.stdout.data) foo hello world! ---------------------------------------------------------------------------- 5.9. How do I access a module written in Python from C? You can get a pointer to the module object as follows: module = PyImport_ImportModule("<modulename>"); If the module hasn't been imported yet (i.e. it is not yet present in sys.modules), this initializes the module; otherwise it simply returns the value of sys.modules["<modulename>"]. Note that it doesn't enter the module into any namespace -- it only ensures it has been initialized and is stored in sys.modules. You can then access the module's attributes (i.e. any name defined in the module) as follows: attr = PyObject_GetAttrString(module, "<attrname>"); Calling PyObject_SetAttrString(), to assign to variables in the module, also works. ---------------------------------------------------------------------------- 5.10. How do I interface to C++ objects from Python? Depending on your requirements, there are many approaches. To do this manually, begin by reading the "Extending and Embedding" document (Doc/ext.tex, see also http://www.python.org/doc/). Realize that for the Python run-time system, there isn't a whole lot of difference between C and C++ -- so the strategy to build a new Python type around a C structure (pointer) type will also work for C++ objects. A useful automated approach (which also works for C) is SWIG: http://www.cs.utah.edu/~beazley/SWIG/. ---------------------------------------------------------------------------- 5.11. mSQLmodule (or other old module) won't build with Python 1.5 (or later) Since python-1.4 "Python.h" will have the file includes needed in an extension module. Backward compatibility is dropped after version 1.4 and therefore mSQLmodule.c will not build as "allobjects.h" cannot be found. The following change in mSQLmodule.c is harmless when building it with 1.4 and necessary when doing so for later python versions: Remove lines: #include "allobjects.h" #include "modsupport.h" And insert instead: #include "Python.h" You may also need to add #include "rename2.h" if the module uses "old names". This may happen with other ancient python modules as well, and the same fix applies. ---------------------------------------------------------------------------- 5.12. I added a module using the Setup file and the make fails! Huh? Setup must end in a newline, if there is no newline there it gets very sad. Aside from this possibility, maybe you have other non-Python-specific linkage problems. ---------------------------------------------------------------------------- 5.13. I want to compile a Python module on my Red Hat Linux system, but some files are missing. Red Hat's RPM for Python doesn't include the /usr/lib/python1.x/config/ directory, which contains various files required for compiling Python extensions. Install the python-devel RPM to get the necessary files. ---------------------------------------------------------------------------- 5.14. What does "SystemError: _PyImport_FixupExtension: module yourmodule not loaded" mean? This means that you have created an extension module named "yourmodule", but your module init function does not initialize with that name. Every module init function will have a line similar to: module = Py_InitModule("yourmodule", yourmodule_functions); If the string passed to this function is not the same name as your extenion module, the SystemError will be raised. ---------------------------------------------------------------------------- -- ----------- comp.lang.python.announce (moderated) ---------- Article Submission Address: python-announce(a)python.org Python Language Home Page: http://www.python.org/ Python Quick Help Index: http://www.python.org/Help.html ------------------------------------------------------------

1 0