Mailman 3 December 2004 - Python-Dev

MinGW And The other Py2.4 issue
by A.B., Khalid 12 Dec '04

12 Dec '04

Nick Craig-Wood wrote: >I'm following this thread with interest because we are considering >embedding Python into this project, and I'm wondering whether we can >cross compile python using mingw (almost certainly by the sound of >it), but probably harder would be to make python module build and >install system work cross-wise. I am following this thread with interest too, because I can certainly report that it is now indeed possible and quite easy to compile not only Python's 2.4 extensions but also Python itself in MinGW with the help of pyMinGW[1]. Indeed there is no need to buy or even download 400MB+ of code to do so. After patching the Python 2.4 source with pyMinGW-24, and issuing the command: make -f python24.mak all We get Python and all its subprojects (socket, ssl, etc.) ready to run: $ /c/temp/py24/python/dist/src/MinGW/python -i Python 2.4.1a0 (#60, Dec 6 2004, 21:05:41) [GCC 3.4.1 (mingw special)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>>import sys >>>sys.winver '2.4' >>> To compile Armin's Psyco 1.3 in my MinGW compiled Python, I simply issued: python setup.py build and then python setup.py bdist_wininst And I had a /dist/psyco-1.3.win32-py2.4.exe file ready to be installed. It works fine at this end (note that I poked myself in the Windows registry to insert the pythoncore version information etc., but that should not be the case with those running the official Windows distribution). I only wonder if extensions created this way will work just as good in the latter. Because if they did, then going down the road of extensive downloads or expensive investment would be truly and totally unnecessary. Regards, Khalid [1] pyMinGW can be found here: http://jove.prohosting.com/iwave/ipython/pyMinGW.html _________________________________________________________________ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

3 2

Supporting Third Party Modules (was The other Py2.4 issue)
by Chui G. Tey 12 Dec '04

12 Dec '04

>Raymond: > >Acceptance for Py2.4 partially hinges on how quickly third party apps >have their binaries updated. > >I wonder if there is anything we can do to help. One good way of helping out is to provide an dynamic loading function that third party modules could access the basic python functions such as PyArgParseTuple, PyString_AsString etc regardless of which python the user is running. This would be similar to the COM approach. You can load all the function pointers into a struct and then call them. Third party modules would link against this DLL independent of which python is being used. Chui

4 9

The other Py2.4 issue
by Raymond Hettinger 12 Dec '04

12 Dec '04

Acceptance for Py2.4 partially hinges on how quickly third party apps have their binaries updated. I wonder if there is anything we can do to help. Raymond -----Original Message----- From: image-sig-bounces(a)python.org [mailto:image-sig-bounces@python.org] On Behalf Of Tuure Laurinolli Sent: Tuesday, December 07, 2004 2:51 AM To: image-sig(a)python.org Subject: [Image-SIG] Re: Python 2.4 Spencer Ogden wrote: > I was wondering if there was a time frame for a windows binary for Py > 2.4. Is it possible to compile the source against 2.4 myself? There didn't seem to be any problems compiling it against 2.4. I managed to get some version up and running, but problems with the required shared libraries(I seem to have many, incompatible versions of jpeg around...) killed my enthusiasm and I returned to 2.3. _______________________________________________ Image-SIG maillist - Image-SIG(a)python.org http://mail.python.org/mailman/listinfo/image-sig

10 21

PEP 338: Executing modules inside packages with '-m'
by Nick Coghlan 12 Dec '04

12 Dec '04

Python 2.4's -m command line switch only works for modules directly on sys.path. Trying to use it with modules inside packages will fail with a "Module not found" error. This PEP aims to fix that for Python 2.5. Previously, posting of a draft version of the PEP to python-dev and python-list didn't actually generate any responses. I'm not sure if that's an indication that people don't see the restriction to top-level modules as a problem (and hence think the PEP is unecessary), or think the extension to handle packages is obvious (and hence see no need to comment). For c.l.p, it could just be a sign that Python 2.4 hasn't been out long enough for anyone to care what I'm yabbering on about :) Anyway, all comments are appreciated (even a simple "Sounds good to me"). Cheers, Nick. *********************************************************************** PEP: 338 Title: Executing modules inside packages with '-m' Version: $Revision: 1.2 $ Last-Modified: $Date: 2004/12/11 20:31:10 $ Author: Nick Coghlan <ncoghlan(a)email.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 16-Oct-2004 Python-Version: 2.5 Post-History: 8-Nov-2004 Abstract ======== This PEP defines semantics for executing modules inside packages as scripts with the ``-m`` command line switch. The proposed semantics are that the containing package be imported prior to execution of the script. Rationale ========= Python 2.4 adds the command line switch ``-m`` to allow modules to be located using the Python module namespace for execution as scripts. The motivating examples were standard library modules such as ``pdb`` and ``profile``. A number of users and developers have requested extension of the feature to also support running modules located inside packages. One example provided is pychecker's ``pychecker.checker`` module. This capability was left out of the Python 2.4 implementation because the appropriate semantics were not entirely clear. The opinion on python-dev was that it was better to postpone the extension to Python 2.5, and go through the PEP process to help make sure we got it right. Scope of this proposal ========================== In Python 2.4, a module located using ``-m`` is executed just as if its filename had been provided on the command line. The goal of this PEP is to get as close as possible to making that statement also hold true for modules inside packages. Prior discussions suggest it should be noted that this PEP is **not** about any of the following: - changing the idiom for making Python modules also useful as scripts (see PEP 299 [1]_). - lifting the restriction of ``-m`` to modules of type PY_SOURCE or PY_COMPILED (i.e. ``.py``, ``.pyc``, ``.pyo``, ``.pyw``). - addressing the problem of ``-m`` not understanding zip imports or Python's sys.metapath. The issues listed above are considered orthogonal to the specific feature addressed by this PEP. Current Behaviour ================= Before describing the new semantics, it's worth covering the existing semantics for Python 2.4 (as they are currently defined only by the source code). When ``-m`` is used on the command line, it immediately terminates the option list (like ``-c``). The argument is interpreted as the name of a top-level Python module (i.e. one which can be found on ``sys.path``). If the module is found, and is of type ``PY_SOURCE`` or ``PY_COMPILED``, then the command line is effectively reinterpreted from ``python <options> -m <module> <args>`` to ``python <options> <filename> <args>``. This includes setting ``sys.argv[0]`` correctly (some scripts rely on this - Python's own ``regrtest.py`` is one example). If the module is not found, or is not of the correct type, an error is printed. Proposed Semantics ================== The semantics proposed are fairly simple: if ``-m`` is used to execute a module inside a package as a script, then the containing package is imported before executing the module in accordance with the semantics for a top-level module. This is necessary due to the way Python's import machinery locates modules inside packages. A package may modify its own __path__ variable during initialisation. In addition, paths may affected by ``*.pth`` files. Accordingly, the only way for Python to reliably locate the module is by importing the containing package and inspecting its __path__ variable. Note that the package is *not* imported into the ``__main__`` module's namespace. The effects of these semantics that will be visible to the executed module are: - the containing package will be in sys.modules - any external effects of the package initialisation (e.g. installed import hooks, loggers, atexit handlers, etc.) Reference Implementation ======================== A reference implementation is available on SourceForge [2]_. In this implementation, if the ``-m`` switch fails to locate the requested module at the top level, it effectively reinterprets the command from ``python -m <script>`` to ``python -m execmodule <script>``. (There is one caveat: when reinterpreted in this way, ``sys.argv[0]`` may not actually contain the filename of ``execmodule``. This only affects ``execmodule`` itself, not the requested module). ``execmodule`` is a proposed standard library module that contains a single function (also called ``execmodule``). When invoked as a script, this module finds and executes the module supplied as the first argument. It adjusts ``sys.argv`` by deleting ``sys.argv[0]`` and replacing the new ``sys.argv[0]`` with the module's filename instead of its Python name. The function ``execmodule`` is like ``execfile``, but uses the Python module namespace to locate the script instead of the filesystem. It has an additional optional argument ``set_argv0`` which causes the filename of the located module to be written to ``sys.argv[0]`` before the module is executed. A hybrid C/Python implementation is used as the Python module is much more flexible and extensible than the equivalent C code would be. It also allows the ``execmodule`` function to be made available. Scripts which execute other scripts (e.g. ``profile``, ``pdb``) have the option to use this function to provide ``-m`` style support for identifying the script to be executed. The Python code for ``execmodule`` has also been posted as a cookbook recipe for Python 2.4 [3]_. Open Issues =========== - choosing a name for the standard library module containing ``execmodule``. The reference implementation uses ``execmodule``. An alternative name proposed on python-dev is ``runpy``. Alternatives ============ The main alternative implementation considered ignored packages' __path__ variables, and looked only in the main package directory. A Python script with this behaviour can be found in the discussion of the ``execmodule`` cookbook recipe [3]_. This approach was not used as it does not meet the main goal of the ``-m`` switch -- to allow the full Python namespace to be used to locate modules for execution. References ========== .. [1] Special __main__() function in modules (http://www.python.org/peps/pep-0299.html) .. [2] Native ``-m`` execmodule support (http://sourceforge.net/tracker/?func=detail&aid=1043356&group_id=5470&atid=… ) .. [3] execmodule Python Cookbook Recipe (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/307772) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: -- Nick Coghlan | ncoghlan(a)email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

3 2

python-dev Summary for 2004-10-16 through 2004-10-31 [draft]
by Brett C. 12 Dec '04

12 Dec '04

Winter Break is upon me which means I have time to catch up on the Summaries. I will definitely be caught up by the end of the month. As for this summary, I will send this out around Wednesday. As always corrections are appreciated. If you feel one of the skipped threads deserves coverage please feel free to write up a summary and I will edit it and include it. -------------------------------------- ===================== Summary Announcements ===================== `Python 2.4`_ final is now out! As I mentioned in the last summary my school schedule this past quarter has been insane. But now I am on Winter Break and will hopefully be able to catch up on my Summary backlog. .. _Python 2.4: http://www.python.org/2.4/ ========= Summaries ========= -------------------------------------------------------- Specifying main functions and calling packages with '-m' -------------------------------------------------------- In my opinion, the new '-m' command line option in Python 2.4 is really handy. But wouldn't it be even handier if you could execute modules in a package? That exact question came up. The reason this kind of thing didn't just go directly into 2.4 was that the semantics are not as obvious nor is it as simple. A PEP is on the way, though. Until then, you can use Nick Coghlan's recipe at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/307772. This also brought up the discussion of being able to specify a 'main' function to take the place of the good old ``if __name__ == "__main__"`` idiom. Some liked the idea of allowing one to define a function named 'main', others '__main__'. But the discussion never went any farther. This will require a PEP to ever even be seriously considered. Contributing threads: - `python-dev Summary for 2004-09-16 through 2004-09-30 [draft] <>`__ - `Magic main functions <>`__ - `Supporting packages on the command line <>`__ ---------------------------- ConfigParser shootout begins ---------------------------- As mentioned in the `last summary`_, a desire for a replacement for ConfigParser has come into being. It seems the ideas are being hashed out in the wiki at http://www.python.org/moin/ConfigParserShootout . Contributing threads: - `ConfigParser shootout, preliminary entry <>`__ - `What are the goals for ConfigParser 3000? <>`__ ---------------------------------------------------- Making pymalloc friendlier to long running processes ---------------------------------------------------- Pymalloc, when a small chunk of memory is requested that is less than 256 bytes, fetches the memory from a pool of previously used memory that is now available. While this speeds up memory allocation since it does not directly involve calling the OS to free up the memory, it does cause issues for long running processes that have at some point requested a large portion of memory. The example in the OP for this whole topic was an app that needs 1GB for about five minutes, but the amount of allocated memory stays at 1 GB since pymalloc does not free it to the OS. This was brought up back in June and is summarized at http://www.python.org/dev/summary/2004-06-01_2004-06-15.html#id17 . No code has been checked in at the moment to change the behavior partially thanks to the difficulty of the problem. Contributing threads: - `Changing pymalloc behaviour for long running processes <>`__ - `Re: Python-Dev Digest, Vol 15, Issue 46 <>`__ ---------------------------- How to get a patch looked at ---------------------------- Often times people have a specific patch that they want reviewed/applied to solve a specific problem they are having. Unfortunately the Python developers have limited time; we just can't get to every patch there in a timely fashion. This can be a problem who *really* need to get a patch in. Enter Martin v. Löwis and his deal to review a specific patch: 1. Review 10 other patches on SF 2. Send an email to python-dev listing the 10 patches that you reviewed and the one patch you want to be reviewed 3. Your specific patch gets reviewed by Martin You can see the exact details of Martin's requirements at http://mail.python.org/pipermail/python-dev/2004-October/049495.html and can also read http://www.python.org/dev/dev_intro.html for ideas on what to do for reviewing. Contributing threads: - `Patches <>`__ ---------------------------------------------- Discussion of including pysqlite in the stdlib ---------------------------------------------- The idea of including pysqlite_ in the stdlib came up once again (not this is the wrapper for SQLite_ and not SQLite itself). The arguments for including the module were the SQLite is most likely used more than Sleepycat and thus deserved a place in the stdlib if bsddb did. It also goes along with Python's idea of "batteries included". Arguments against started with the idea of sanctioning pysqlite over other SQLite wrappers did not seem proper. People also thought that including bsddb might not be correct anymore and thus should not be used as a basis for including pysqlite. This all then led into a discussion about package management and how to simplify extending what can be installed at compile-time. The idea of pushing PyPI_ came up as well as revising `PEP 206`_. .. _pysqlite: http://pysqlite.org/ .. _SQLite: http://www.sqlite.org/ .. _PyPI: http://www.python.org/pypi .. _PEP 206: http://www.python.org/peps/pep-0206.html Contributing threads: - `SQLite module for Python 2.5 <>`__ =============== Skipped Threads =============== - Python 2.4b1 on win32 installer bug? - test_subprocess 2.4b1 fails on FreeBSD 5.2 - adding Py{String|Unicode}_{Lower|Upper} and fixing CreateProcess in _subprocess.pyd and PyWin32 - IPv6 bug in 2.3.4?? - logging needs better documentation - bsddb todo for someone - auto-upgrade db format - Tests broken on Windows - import w/options - Adding a developer People are only added as a developer if they ask for it and have shown they are going to stick around - audiotest.au and possible copyright issues? - Re: [Python-checkins] python/dist/src/Pythoncompile.c, 2.330, 2.331 - Bug [ 959379 ] Implicit close() should check for errors - weakref callback vs gc vs threads

2 1

Re: [Python-Dev] Re: 2.4 news reaches interesting places
by Nick Coghlan 12 Dec '04

12 Dec '04

Raymond Hettinger wrote: <some sensible things> Makes sense to me - I look forward to seeing the next version. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net

1 0

removing aclocal.m4
by Neal Norwitz 11 Dec '04

11 Dec '04

Can we remove aclocal.m4? The last log message states: fix for bug #811160 - autoconf vs. hp/ux system header files. also applied to release23-maint. Note that aclocal.m4 can go away when autoconf 2.58 is out. And configure says: # Generated by GNU Autoconf 2.59 for python 2.5. Neal

2 1

PyOS_InputHook enhancement proposal
by Michiel Jan Laurens de Hoon 10 Dec '04

10 Dec '04

PyOS_InputHook is a pointer to a function that is called periodically (ten times per second) while Python is idle, for example, when waiting for a user command. Python C extension modules can set this pointer to a hook function defined in the extension module. For example, _tkinter.c makes use of PyOS_InputHook to get messages delivered to its widgets. A problem arises when two or more extension modules want to set PyOS_InputHook. For example, the scientific plotting package pygist needs PyOS_InputHook to get messages delivered to its graphics windows, and may therefore conflict with the Python GUI IDLE, which uses Tkinter. Chaining won't work, as it will fail when an extension module wants to remove its hook function. My suggestion is therefore to replace PyOS_InputHook by two functions PyOS_AddInputHook and PyOS_RemoveInputHook, and let Python keep track of which hooks are installed. This way, an extension module can add a hook function without having to worry about other extension modules trying to use the same hook. Any comments? Would I need to submit a PEP for this proposal? --Michiel, U Tokyo. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

4 3

RE: [Python-Dev] Re: 2.4 news reaches interesting places
by Delaney, Timothy C (Timothy) 10 Dec '04

10 Dec '04

Michael Hudson wrote: > Anthony's Australian, people expect this sort of thing from him :) As another Australian, I think that "Making Python Not Suck" implies that if you don't do extra things, Python Sucks. This is not a good thing IMO. "Making Python Suck Less" would be even worse. How about "Python - You Can Have Your Cake And Eat It Too". Tim Delaney

4 3

Proposed Mac OS X change for 2.3.5 and 2.4.1 - MACOSX_DEPLOYMENT_TARGET vs. Python, round two, fight!
by Bob Ippolito 10 Dec '04

10 Dec '04

I've included a lot of background information here, if you just want to know the details of the proposed change, skip to the bottom. As some of you may know, Python 2.4's configure script and distutils has some tricky behavior with regard to the ``MACOSX_DEPLOYMENT_TARGET`` environment variable on Mac OS X 10.3 and later. Unless otherwise specified, assume that I am talking about a build environment of Mac OS X 10.3 or later. This behavior is as follows: If ``MACOSX_DEPLOYMENT_TARGET`` is not set during configure: 1. distutils builds modules with ``-F/Python/Installation/Location -framework Python`` as per usual 2.3.x behavior if it is also not set 2. If ``MACOSX_DEPLOYMENT_TARGET`` is set during a later run of distutils, then distutils complains that "10.3" mismatches the configure time setting of "" This Python framework has the following misfeatures: 1. All your extensions are dependent on the installation location of this particular Python 2. This installation of Python 2.4 *may break the building of extensions* for any previous version of Python that uses the same distutils behavior. It will certainly break them if they are installed to the same framework directory, but if the version of Python 2.3 is early enough, such as the stock 2.3.0, it will break that too. Also, any future version of Python installed to the same frameworks directory *will break the building of extensions* for this Python installation! 3. The Python framework will not be compatible with versions of Mac OS X earlier than 10.3 anyway due to changes in libSystem! This is stupid, and it should NOT be default behavior! If ``MACOSX_DEPLOYMENT_TARGET`` is set to "10.3" or higher during configure: 1. distutils builds modules with ``-undefined dynamic_lookup`` 2. If ``MACOSX_DEPLOYMENT_TARGET`` is set to something other than "10.3", or unset, it will complain that the current setting mismatches the configure setting of "10.3" The features: 1. All your extensions are independent of the Python installation location, and are thus compatible with any other Python with the same major version. 2. This installation of Python 2.4 will still, unavoidably, break the building of extensions for any previous version of Python using the bad distutils behavior mentioned above. This installation is not susceptible to breakage, however. The misfeatures: 1. You need to have ``MACOSX_DEPLOYMENT_TARGET`` set during configure, most people don't know to do this. 2. You also need to have ``MACOSX_DEPLOYMENT_TARGET`` set when using distutils. Even less people know to do this. The solution to this is to have the following behavior instead: 1. If no ``MACOSX_DEPLOYMENT_TARGET`` is set during configure, and the build machine is Mac OS X 10.3 or later, then set it to "10.3". 2. If no ``MACOSX_DEPLOYMENT_TARGET`` is set during distutils, but it WAS set during configure, then set it to the configure time value. 3. Otherwise, if it is set to a LOWER value, then fail. Checking for an exact match is bad, because the user or extension author should be free to build a particular extension using 10.4 specific features against a Python that is 10.3+ compatible. For a build machine running Mac OS X 10.2 or earlier, it should ignore all of this behavior as per usual. Unless anyone has a reasonable objection to this proposed solution, then we should make sure it goes into Python 2.4.1 and Python 2.3.5. These threads might also be of interest: http://mail.python.org/pipermail/pythonmac-sig/2004-November/012192.html http://mail.python.org/pipermail/pythonmac-sig/2004-December/012237.html http://mail.python.org/pipermail/pythonmac-sig/2004-December/012275.html -bob

1 0