Mailman 3 September 2003 - Python-announce-list

ZODB 3.2b3 released
by Jeremy Hylton 23 Sep '03

23 Sep '03

We released ZODB 3.2b2 on Tuesday evening. You can find it at the usual place: http://www.zope.org/Products/ZODB3.2 This release contains the last new work we plan to do before the ZODB 3.2 final release. We have fixed several bugs, including a deadlock problem that had been reported against 3.2b2. ZODB 3.2 has a number of new features and improvements over ZODB 3.1: - improve performance and stability of ZEO - new ZEO authentication protocol - new configuration language, ZConfig, for databases, storages, and ZEO servers - many bug fixes Please give this new release a try. We hope to have a final release soon, and bugs reported soon are more likely to get fixed before the final release. I've included the NEWS.txt summary of the most recent changes. Jeremy What's new in ZODB3 3.2 beta 3 ============================== Release date: 23-Sep-2003 Note: The changes listed for this release include changes also made in ZODB 3.1.x releases and ported to the 3.2 release. This version of ZODB 3.2 is not compatible with Python 2.1. Early versions were explicitly designed to be compatible with Zope 2.6. That plan has been dropped, because Zope 2.7 is already in beta release. Several of the classes in ZEO and ZODB now inherit from object, making them new-style classes. The primary motivation for the change was to make it easier to debug memory leaks. We don't expect any behavior to change as a result. A new feature to allow removal of connection pools for versions was ported from Zope 2.6. This feature is needed by Zope to avoid denial of service attacks that allow a client to create an arbitrary number of version pools. Fixed several critical ZEO bugs. - If several client transactions were blocked waiting for the storage and one of the blocked clients disconnected, the server would attempt to restart one of the other waiting clients. Since the disconnected client did not have the storage lock, this could lead to deadlock. It could also cause the assertion "self._client is None" to fail. - If a storage server fails or times out between the vote and the finish, the ZEO cache could get populated with objects that didn't make it to the storage server. - If a client loses its connection to the server near the end of a transaction, it is now guaranteed to get a ClientDisconnected error even if it reconnects before the transaction finishes. This is necessary because the server will always abort the transaction. In some cases, the client would never see an error for the aborted transaction. - In tpc_finish(), reordered the calls so that the server's tpc_finish() is called (and must succeed) before we update the ZEO client cache. - The storage name is now prepended to the sort key, to ensure a unique global sort order if storages are named uniquely. This can prevent deadlock in some unusual cases. Fixed several serious flaws in the implementation of the ZEO authentication protocol. - The smac layer would accept a message without a MAC even after the session key was established. - The client never initialized its session key, so it never checked incoming messages or created MACs for outgoing messags. - The smac layer used a single HMAC instance for sending and receiving messages. This approach could only work if client and server were guaranteed to process all messages in the same total order, which could only happen in simple scenarios like unit tests. Fixed a bug in ExtensionClass when comparing ExtensionClass instances. The code could raise RuntimeWarning under Python 2.3, and produce incorrect results on 64-bit platforms. Fixed bug in BDBStorage that cause lead to DBRunRecoveryErrors when a transaction was aborted after performing operations like commit version or undo that create new references to existing pickles. Fixed a bug in Connection.py that caused it to fail with an AttributeError if close() was called after the database was closed. The test suite leaves fewer log files behind, although it still leaves a lot of junk. The test.py script puts each tests temp files in a separate directory, so it is easier to see which tests are causing problems. Unfortunately, it is still to tedious to figure out why the identified tests are leaving files behind. This release contains the latest and greatest version of the BDBStorage. This storage has still not seen testing in a production environment, but it represents the current best design and most recent code culled from various branches where development has occurred. The Tools directory contains a number of small improvements, a few new tools, and README.txt that catalogs the tools. Many of the tools are installed by setup.py; those scripts will now have a #! line set automatically on Unix. Fixed bugs in Tools/repozo.py, including a timing-dependent one that could cause the following invocation of repozo to do a full backup when an incremental backup would have sufficed. A pair of new scripts from Jim Fulton can be used to synthesize workloads and measure ZEO performance: see zodbload.py and zeoserverlog.py in the Tools directory. Note that these require Zope. Tools/checkbtrees.py was strengthened in two ways: - In addition to running the _check() method on each BTree B found, BTrees.check.check(B) is also run. The check() function was written after checkbtrees.py, and identifies kinds of damage B._check() cannot find. - Cycles in the object graph no longer lead to unbounded output. Note that preventing this requires remembering the oid of each persistent object found, which increases the memory needed by the script.

1 0

Python Database Objects (PDO) 1.0.1 Released
by Bryan J Gudorf 23 Sep '03

23 Sep '03

We at NeuroKode Labs, LLC, are proud to announce the release of Python Database Objects (PDO) 1.0.1. The current release adds support for PostgreSQL via the pgdb Module. As released previously: Python Database Objects (PDO) provides an easy to use Object Oriented API for database developers. PDO utilizes DB-API modules for database access, but allows for a Common Object Oriented API across RDBMS. Thus, PDO can be thought of as a 'wrapper' around the DB-API and database specific modules. PDO offers a unique interface which includes such features as column access by name, forward and backward movement through a result set and much, much more. Downloads for Python Database Objects are available on SourceForge.Net or for more information please visit pdo.neurokode.com. ~Bryan J Gudorf NeuroKode Labs, LLC

1 0

Python training in Colorado, Oct 7-9
by lutz＠rmi.net 22 Sep '03

22 Sep '03

I am holding another 3-day Python class in Longmont, Colorado, on October 7-9. This is a public class, open to individual enrollments, and covers the same topics as the on-site classes I teach. For more details, please see this page: http://www.rmi.net/~lutz/oct03-longmont-class.html Thanks for your interest. --Mark Lutz (http://www.rmi.net/~lutz)

1 0

ANN: PyTables 0.7.2 - A hierarchical database
by Francesc Alted 22 Sep '03

22 Sep '03

Announcing PyTables 0.7.2 ------------------------- PyTables is a hierarchical database package designed to efficently manage very large amounts of data. PyTables is built on top of the HDF5 library and the numarray package. It features an object-oriented interface that, combined with natural naming and C-code generated from Pyrex sources, makes it a fast, yet extremely easy to use tool for interactively save and retrieve large amounts of data. Besides, it provides flexible indexed access on disk to anywhere in the data you want to go. On this release you will not find any exciting new features. It is mainly a maintenance release where the next issues has been addressed: - a memory leak was fixed - memory needs is being lowered - much faster opening of files - done some important optimizations in table reads More in detail: What's new ----------- - Fixed a nasty memory leak located on the C libraries (it was happening during HDF5 attribute writes). After that, the memory consumption when using large object trees has dropped quite a bit. However, there remains some small leaks that has been tracked down to the underlying numarray library. These leaks has been reported, and hopefully they should be fixed more sooner than later. - Table buffers are built dinamically now, so if Tables are not accessed for reading or writing this memory will not be booked. This will help to reduce the memory consumption. - The opening of files with lots of nodes has been accelerated between a factor 2 or 3. For example, a file with 10 groups and 3000 tables that takes 9.3 seconds to open in 0.7.1, now takes only 2.8 seconds. - The Table.read() method has been refactored and optimized and some parts of its code has been moved to Pyrex. In particular, in the special case of step=1, up to a factor 5 of speedup (reaching 160 MB/s on a Pentium4 @ 2 GHz) when reading table contents can be achieved now. - Done some cosmetic changes in the user manual, but, as no new features has been added, you won't need to read the manual again :-) What is a table? ---------------- A table is defined as a collection of records whose values are stored in fixed-length fields. All records have the same structure and all values in each field have the same data type. The terms "fixed-length" and "strict data types" seems to be quite a strange requirement for an language like Python, that supports dynamic data types, but they serve a useful function if the goal is to save very large quantities of data (such as is generated by many scientific applications, for example) in an efficient manner that reduces demand on CPU time and I/O resources. What is HDF5? ------------- For those people who know nothing about HDF5, it is is a general purpose library and file format for storing scientific data made at NCSA. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic constructs, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids. You can also mix and match them in HDF5 files according to your needs. Platforms --------- I'm using Linux as the main development platform, but PyTables should be easy to compile/install on other UNIX machines. This package has also passed all the tests on a UltraSparc platform with Solaris 7 and Solaris 8. It also compiles and passes all the tests on a SGI Origin2000 with MIPS R12000 processors and running IRIX 6.5. Regarding Windows platforms, PyTables has been tested with Windows 2000 and Windows XP, but it should also work with other flavors. An example? ----------- For online code examples, have a look at http://pytables.sourceforge.net/tut/tutorial1-1.html and http://pytables.sourceforge.net/tut/tutorial1-2.html Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Have fun! -- Francesc Alted

1 0

python-dev Summary for 2003-09-01 through 2003-09-15
by Brett C. 21 Sep '03

21 Sep '03

python-dev Summary for 2003-09-01 through 2003-09-15 ++++++++++++++++++++++++++++++++++++++++++++++++++++ This is a summary of traffic on the `python-dev mailing list`_ from September 1, 2003 through September 15, 2003. It is intended to inform the wider Python community of on-going developments on the list. To comment on anything mentioned here, just post to `comp.lang.python`_ (or email python-list(a)python.org which is a gateway to the newsgroup) with a subject line mentioning what you are discussing. All python-dev members are interested in seeing ideas discussed by the community, so don't hesitate to take a stance on something. And if all of this really interests you then get involved and join `python-dev`_! This is the twenty-fifth summary written by Brett Cannon (with school looming on the horizon). All summaries are archived at http://www.python.org/dev/summary/ . Please note that this summary is written using reStructuredText_ which can be found at http://docutils.sf.net/rst.html . Any unfamiliar punctuation is probably markup for reST_ (otherwise it is probably regular expression syntax or a typo =); you can safely ignore it, although I suggest learning reST; it's simple and is accepted for `PEP markup`_ and gives some perks for the HTML output. Also, because of the wonders of programs that like to reformat text, I cannot guarantee you will be able to run the text version of this summary through Docutils_ as-is unless it is from the original text file. .. _PEP Markup: http://www.python.org/peps/pep-0012.html The in-development version of the documentation for Python can be found at http://www.python.org/dev/doc/devel/ and should be used when looking up any documentation on something mentioned here. PEPs (Python Enhancement Proposals) are located at http://www.python.org/peps/ . To view files in the Python CVS online, go to http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ . Reported bugs and suggested patches can be found at the SourceForge_ project page. .. _python-dev: http://www.python.org/dev/ .. _SourceForge: http://sourceforge.net/tracker/?group_id=5470 .. _python-dev mailing list: http://mail.python.org/mailman/listinfo/python-dev .. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python .. _Docutils: http://docutils.sf.net/ .. _reST: .. _reStructuredText: http://docutils.sf.net/rst.html .. contents:: .. _last summary: http://www.python.org/dev/summary/2003-08-16_2003-08-31.html ===================== Summary Announcements ===================== Summary is nice and short this time. More people went on vacation (although Martin came back and so that helped to pick up the traffic a little). I skipped covering some threads that are due for PEPs since those will be more in-depth then anything I write. I should give fair warning that starting on the 22nd of September I will begin school. I am hoping that it will not be difficult and I will have a carefree time in school. But if my workload becomes a burden the Summaries might suffer. I hope they don't, though, and I am going to do my best to make sure that doesn't happen. ========= Summaries ========= --------------------------------------- If It Isn't Documented, Is It a Secret? --------------------------------------- Some undocumented methods in readline were discovered by Philip Eby. He asked if this was on purpose and whether he should submit a bug report on the lack of documentation. Guido told him to go ahead and submit the bug report. If you ever find something undocumented in the Python source code and there is nothing suggesting the code is meant to be hidden from the user (it's for internal use, a comment says it is experimental, etc.), then please file a bug report! Contributing threads: - `Undocumented functions in 'readline' module <http://mail.python.org/pipermail/python-dev/2003-September/037922.html>`__ ----------------------------------------------------------------------------------------- "You are more trouble than you are worth!", screamed the micro release to the new feature ----------------------------------------------------------------------------------------- The topic of what should be backported for micro releases (with Python 2.3.1 looming on the horizon) came up. Originally minor improvements were allowed in if they didn't break backwards compatibility. And of course bugfixes have been as well as long as they didn't break code that came to depend on the buggy behavior (really depends on how long the bugginess has been there and how well-known it is). But then the point of OS X 10.3 possibly becoming the largest install base of Python of any version (it will be 2.3) came up. With rough estimates being thrown around of 5 million installs in about a year's time, the point that making it difficult to run Python 2.3.x code on that size of an install base would be bad. For instance, if some small new feature was added to 2.3.1 then any code using that feature would not be able to run on a virgin install of OS X 10.3 (and considering that this is Mac it should not be expected that most users will want to, let alone know how, to add a secondary install of Python since the original is used by the OS and thus should not be overwritten). It looks like a more conservative patching scheme will be taken with micro releases. Bytecode will also continue to work between releases. Guido has always unofficially held that position but now it has been vocalized. Contributing threads: - `Documenting branch policy <http://mail.python.org/pipermail/python-dev/2003-September/037955.html>`__ ---------------------------------------------- PyPy "Berlin" sprint (29th Sep - 4th Oct 2003) ---------------------------------------------- Just what the title says: there is going to be a sprint_ for PyPy_ in Berlin from September 29 to October 4. Read the email for more specific details on goals and such. .. _sprint: http://www.python.org/cgi-bin/moinmoin/SprintPlan .. _PyPy: http://codespeak.net/pypy/index.cgi?home Contributing threads: - `PyPy "Berlin" sprint (29th Sep - 4th Oct 2003) <http://mail.python.org/pipermail/python-dev/2003-September/037961.html>`__ -------------------------------- Away with you ambiguous imports! -------------------------------- A discussion on how to create a hierarchy of loggers in the standard library led to the idea of having a way to cause an import to come directly from the standard library and thus remove any ambiguity from a relative import grabbing a similarly named module from the local package. Barry Warsaw said he was thinking of writing a PEP on this idea. Contributing threads: - `Consistent logging in the standard library <http://mail.python.org/pipermail/python-dev/2003-September/038000.html>`__ --------------------------------------------------- Quick: want to give a Python talk at Linuxworld NY? --------------------------------------------------- If you are interested in what the title asks, then read the email for some details. Contributing threads: - `Quick: want to give a Python talk at Linuxworld NY? <http://mail.python.org/pipermail/python-dev/2003-September/037993.html>`__ ------------------------------------------- Be careful about saying something is "easy" ------------------------------------------- A word of advice about saying something is "easy" on python-dev: if you say that you should make sure you can back up that claim because otherwise you will be told to write the "easy" patch and that will be the end of the discussion. Contributing threads: - `Making python C-API thread safe (try 2) <http://mail.python.org/pipermail/python-dev/2003-September/038007.html>`__ -------------------------- Parsing dates for datetime -------------------------- There was some talk about coming up with some code to parse dates for the datetime module. It was kind of hand-wavy since there are multiples chunks of code that can do that in the standard library. If you have an opinion on this (or anything for that matter), make sure to make a post to `comp.lang.python`_ about it. Contributing threads: - `datetime issues <http://mail.python.org/pipermail/python-dev/2003-September/038042.html>`__

1 0

ANN: ClientForm 0.1.8b released
by jjl＠pobox.com 21 Sep '03

21 Sep '03

http://wwwsearch.sourceforge.net/ClientForm/ http://wwwsearch.sourceforge.net/ClientForm/src/README-0_1_8b.html Second (and last, I hope) beta release of 0.1.x. Changes from 0.1.7a to 0.1.8b: * Interface change (sorry): id is now directly supported. Controls have an id attribute, and appropriate HTMLForm methods have an id argument. This will only affect code that uses the positional arguments that come after the 'kind' argument of the various HTMLForm methods. * Interface change: BUTTON/BUTTON now has type "buttonbutton" (was "button") to prevent clash with type of INPUT/BUTTON (was and is "button"). Both types of control are ignored anyway (ie. represented by IgnoreControl), so it's unlikely any code is affected. * SubmitButton value now defaults to "", so it is successful even when no value is given in the HTML. * File upload bugs fixed. * Minor bug fixes. Requires Python >= 1.5.2. ClientForm is a Python module for handling HTML forms on the client side, useful for parsing HTML forms, filling them in and returning the completed forms to the server. It has developed from a port of Gisle Aas' Perl module HTML::Form, from the libwww-perl library, but the interface is not the same. Simple example: from urllib2 import urlopen from ClientForm import ParseResponse forms = ParseResponse(urlopen("http://www.acme.com/form.html")) form = forms[0] print form form["author"] = "Gisle Aas" # form.click returns a urllib2.Request object # (see HTMLForm.click_request_data.__doc__ if you're not using urllib2) response = urlopen(form.click("Thanks")) John

1 0

ANN: pyTerra: TerraServer module for Python
by Howard Butler 20 Sep '03

20 Sep '03

pyTerra is a Python module that allows you to make requests to Microsoft's TerraServer. With it, you can download cartographic images for any almost any geographic extent in the conterminous US. It mimics the SOAP API provided by the TerraServer located at <http://terraservice.net/TerraService.asmx>. A helper class is also provided that reduces the interaction required to defining a bounding box and writing the image out to the file system. A source distribution and Windows binary is provided. See <http://hobu.biz/software/pyTerra> for more information and download. Howard Butler

1 0

[ANN] imgSeek 0.8.2
by nieder＠mail.ru 20 Sep '03

20 Sep '03

mgSeek ------- imgSeek is a photo collection manager and viewer with content-based search and many other features. The query is expressed either as a rough sketch painted by the user or as another image you supply (or an image in your collection). You may also do slideshows, generate web photo albums, edit image metadata including EXIF and IPTC data, organize images into a keyword hierarchy, and more. Changes ------- A bug related to settings prevented users with python2.3 from viewing images on the preview window, so 0.8.2 sole purpose is to fix that. Not much development effort has been put lately on new imgSeek features, so expect only bugfixes to be released in the next few months. Meanwhile we've been working on the imgSeek-net sub-project (http://imgseek.sourceforge.net/net/ -- 'imgseek-net' cvs module) and on the 'SQL' cvs branch of imgSeek, whose long term objective would be the usage of popular RDBMS as a transparent replacement for the current implementation of a grouping and metadata database on top of python's 'marshal' module. Work on the imgSeek-lite (http://imgseek.sourceforge.net/lite/ -- 'imgseek-lite' cvs module) subproject will also slowdown, as most of the code there will evolve into the 2nd imgseek-net client prototype. Requires -------- - Python 2.2.x, QT 3.x and PyQT 3.5. (3.4 should work) - ImageMagick development files or QT development files. Recommended: - Python Imaging Library. Links ----- Download: http://sourceforge.net/project/showfiles.php?group_id=70373 Homepage: http://imgseek.sourceforge.net/ Screenshots: http://imgseek.sourceforge.net/sshot/ Complete ChangeLog: http://imgseek.sourceforge.net/changelog.html Best regards, --- rnc <nieder(a)mail.ru>

1 0

ANN: Vision Egg 0.9.9 - Python/OpenGL visual stimulus generation
by andrew.straw＠adelaide.edu.au 20 Sep '03

20 Sep '03

The Vision Egg is a powerful, flexible, and free way to produce stimuli for vision research experiments available at: http://www.visionegg.org/ The Vision Egg is a high level interface between Python and OpenGL. In addition to methods for automatic generation of traditional visual stimuli such as sinusoidal gratings and random dot patterns, it has a number of functions for moving numeric data, images, movies, and text to and from your video card. Therefore, it is also useful for anyone wishing to make use of the features of today's graphics cards. Features ======== * Perform experiments using an inexpensive PC and standard consumer graphics card * Perform experiments using a graphics workstation if special features needed * Data acquisition and other realtime hardware control capabilities useful in electrophysiology and fMRI experiments, including gaze-contingent stimuli * Dynamically generated stimuli can be changed in realtime via software or external hardware * Produce traditional stimuli to replace legacy systems * Produce stimuli not possible using other hardware * Demo programs to get you started right away * Run stimuli on your laptop - great for talks * Free, open-source software It runs on anything from cheap PCs to expensive special hardware for special needs. For example, running on some platforms, such as SGI workstations, the Vision Egg has a 10-bit luminance dynamic range (both pixel depth and DAC) and precise frame-by-frame control. The Vision Egg is open source software (GNU LGPL). - Andrew Straw

1 0

ANN: EmPy 3.1.1 -- A powerful and robust templating system for Python
by Erik Max Francis 20 Sep '03

20 Sep '03

Summary A powerful and robust templating system for Python. Overview EmPy is a system for embedding Python expressions and statements in template text; it takes an EmPy source file, processes it, and produces output. This is accomplished via expansions, which are special signals to the EmPy system and are set off by a special prefix (by default the at sign, '@'). EmPy can expand arbitrary Python expressions and statements in this way, as well as a variety of special forms. Textual data not explicitly delimited in this way is sent unaffected to the output, allowing Python to be used in effect as a markup language. Also supported are callbacks via hooks, recording and playback via diversions, and dynamic, chainable filters. The system is highly configurable via command line options and embedded commands. Expressions are embedded in text with the '@(...)' notation; variations include conditional expressions with '@(...?...!...)' and the ability to handle thrown exceptions with '@(...$...)'. As a shortcut, simple variables and expressions can be abbreviated as '@variable', '@object.attribute', '@function(arguments)', '@sequence[index]', and combinations. Full-fledged statements are embedded with '@{...}'. Control flow in terms of conditional or repeated expansion is available with '@[...]'. A '@' followed by a whitespace character (including a newline) expands to nothing, allowing string concatenations and line continuations. Comments are indicated with '@#' and consume the rest of the line, up to and including the trailing newline. '@%' indicate "significators," which are special forms of variable assignment intended to specify per-file identification information in a format which is easy to parse externally. Context name and line number changes can be done with '@?' and '@!' respectively. Escape sequences analogous to those in C can be specified with '@\...', and finally a '@@' sequence expands to a single literal at sign. Getting the software The current version of empy is 3.1.1. The latest version of the software is available in a tarball here: http://www.alcyone.com/software/empy/empy-latest.tar.gz. The official URL for this Web site is http://www.alcyone.com/software/empy/. Requirements EmPy should work with any version of Python from 1.5.2 onward. It has been tested with all major versions of CPython from 1.5 up, and Jython from 2.0 up (using Java runtimes 1.3 and 1.4). The included test script is intended to run on Unix-like systems with a Bourne shell. License This code is released under the GPL. .... Release history (since 3.1) - 3.1.1; 2003 Sep 20. Added literal '@"..."' markup; added --pause-at-end command line option; fixed improper globals collision error via the 'sys.stdout' proxy.

1 0