Mailman 3 March 2004 - Python-announce-list

[ANN] PyTables 0.8 is out!
by Francesc Alted 08 Mar '04

08 Mar '04

[Ooops, I should have published this here past week, but I had a mistake] I'm happy to announce the availability of PyTables 0.8. PyTables is a hierarchical database package designed to efficiently manage very large amounts of data. PyTables is built on top of the HDF5 library and the numarray package. It features an object-oriented interface that, combined with natural naming and C-code generated from Pyrex sources, makes it a fast, yet extremely easy-to-use tool for interactively saving and retrieving different kinds of datasets. It also provides flexible indexed access on disk to anywhere in the data. In this release you will find: - Variable Length Arrays (VLA's) for saving a collection of variable length of elements in each row of a dataset. - Enlargeable Arrays (EA's) for enlarge homogeneous datasets on disk. - Powerful replication capabilities, ranging from single leaves up to complete hierarchies. - With the introduction of the UnImplemented class, greatly improved HDF5 native file import capabilities. - Two new useful utilities: ptdump & ptrepack. - Improved documentation (with the help of Scott Prater). - Enhanced platform support. New platforms: MacOSX, FreeBSD, Linux64, IRIX64 (yes, a clean 64-bit port is there) and probably more. - More test units (now exceeding 800). - Many other minor improvements. Besides, using PyTables 0.8 a new record on data size has been achieved: 5.6 TB (~ 1000 DVD's) in one single file (!). See the gory details in: http://pytables.sf.net/html/StressTests.html. More in detail: What's new ----------- - The new VLArray class enables you to store large lists of rows containing variable numbers of elements. The elements can be scalars or fully multimensional objects, in the PyTables tradition. This class supports two special objects as rows: Unicode strings (UTF-8 codification is used internally) and generic Python objects (through the use of cPickle). - The new EArray class allows you to enlarge already existing multidimensional homogeneous data objects. Consider it an extension of the already existing Array class, but with more functionality. Online compression or other filters can be applied to EArray instances, for example. Another nice feature of EA's is their support for fully multidimensional data selection with extended slices. You can write "earray[1,2:3,...,4:200]", for example, to get the desired dataset slice from the disk. This is implemented using the powerful selection capabilities of the HDF5 library, which results in very highly efficient I/O operations. The same functionality has been added to Array objects as well. - New UnImplemented class. If a dataset contains unsupported datatypes, it will be associated with an UnImplemented instance, then inserted into to the object tree as usual. This allows you to continue to work with supported objects while retaining access to attributes of unsupported datasets. This has changed from previous versions, where a RuntimeError occurred when an unsupported object was encountered. The combination of the new UnImplemented class with the support for new datatypes will enable PyTables to greatly increase the number of types of native HDF5 files that can be read and modified. - Boolean support has been added for all the Leaf objects. - The Table class has now an append() method that allows you to save large buffers of data in one go (i.e. bypassing the Row accessor). This can greatly improve data gathering speed. - The standard HDF5 shuffle filter (to further enhance the compression level) is supported. - The standard HDF5 fletcher32 checksum filter is supported. - As the supported number of filters is growing (and may be further increased in the future), a Filters() class has been introduced to handle filters more easily. In order to add support for this class, it was necessary to make a change in the createTable() method that is not backwards compatible: the "compress" and "complib" parameters are deprecated now and the "filters" parameter should be used in their place. You will be able to continue using the old parameters (only a Deprecation warning will be issued) for the next few releases, but you should migrate to the new version as soon as possible. In general, you can easily migrate old code by substituting code in its place: tbl = fileh.createTable(group, 'table', Test, '', complevel, complib) should be replaced by tbl = fileh.createTable(group, 'table', Test, '', Filters(complevel, complib)) - A copy() method that supports slicing and modification of filtering capabilities has been added for all the Leaf objects. See the User's Manual for more information. - A couple of new methods, namely copyFile() and copyChilds(), have been added to File class, to permit easy replication of complete hierarchies or sub-hierarchies, even to other files. You can change filters during the copy process as well. - Two new utilities has been added: ptdump and ptrepack. The utility ptdump allows the user to examine the contents of PyTables files (both metadata and actual data). The powerful ptrepack utility lets you selectively copy (portions of) hierarchies to specific locations in other files. It can be also used as an importer for generic HDF5 files. - The meaning of the stop parameter in read() methods has changed. Now a value of 'None' means the last row, and a value of 0 (zero) means the first row. This is more consistent with the range() function in python and the __getitem__() special method in numarray. - The method Table.removeRows() is no longer limited by table size. You can now delete rows regardless of the size of the table. - The "numarray" value has been added to the flavor parameter in the Table.read() method for completeness. - The attributes (.attr instance variable) are Python properties now. Access to their values is no longer lazy, i.e. you will be able to see both system or user attributes from the command line using the tab-completion capability of your python console (if enabled). - Documentation has been greatly improved to explain all the new functionality. In particular, the internal format of PyTables is now fully described. You can now build "native" PyTables files using any generic HDF5 software by just duplicating their format. - Many new tests have been added, not only to check new functionality but also to more stringently check existing functionality. There are more than 800 different tests now (and the number is increasing :). - New platforms supported: PyTables has been compiled and tested under GNU/Linux32 (Intel), GNU/Linux64 (AMD Opteron and Alpha), Win32 (Intel), MacOSX (PowerPC), FreeBSD (Intel), Solaris (6, 7, 8 and 9 with UltraSparc), IRIX64 (IRIX 6.5 with R12000) and it probably works in many more architectures. In particular, release 0.8 is the first one that provides a relatively clean porting to 64-bit platforms. - As always, some bugs have been solved (especially bugs that occur when deleting and/or overwriting attributes). - And last, but definitely not least, a new donations section has been added to the PyTables web site (http://sourceforge.net/projects/pytables, then follow the "Donations" tag). If you like PyTables and want this effort to continue, please, donate! Where PyTables can be applied? ------------------------------ PyTables is not designed to work as a relational database competitor, but rather as a teammate. If you want to work with large datasets of multidimensional data (for example, for multidimensional analysis), or just provide a categorized structure for some portions of your cluttered RDBS, then give PyTables a try. It works well for storing data from data acquisition systems (DAS), simulation software, network data monitoring systems (for example, traffic measurements of IP packets on routers), working with very large XML files or as a centralized repository for system logs, to name only a few possible uses. What is a table? ---------------- A table is defined as a collection of records whose values are stored in fixed-length fields. All records have the same structure and all values in each field have the same data type. The terms "fixed-length" and "strict data types" seem to be quite a strange requirement for a language like Python that supports dynamic data types, but they serve a useful function if the goal is to save very large quantities of data (such as is generated by many scientific applications, for example) in an efficient manner that reduces demand on CPU time and I/O resources. What is HDF5? ------------- For those people who know nothing about HDF5, it is a general purpose library and file format for storing scientific data made at NCSA. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic constructs, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids. You can also mix and match them in HDF5 files according to your needs. Platforms --------- I'm using Linux (Intel 32-bit) as the main development platform, but PyTables should be easy to compile/install on many other UNIX machines. This package has also passed all the tests on a UltraSparc platform with Solaris 7 and Solaris 8. It also compiles and passes all the tests on a SGI Origin2000 with MIPS R12000 processors, with the MIPSPro compiler and running IRIX 6.5. It also runs fine on Linux 64-bit platforms, like an AMD Opteron running SuSe Linux Enterprise Server. It has also been tested in MacOSX platforms (10.2 but should also work on newer versions). Regarding Windows platforms, PyTables has been tested with Windows 2000 and Windows XP (using the Microsoft Visual C compiler), but it should also work with other flavors as well. An example? ----------- For online code examples, have a look at http://pytables.sourceforge.net/html/tut/tutorial1-1.html and, for newly introduced Variable Length Arrays: http://pytables.sourceforge.net/html/tut/vlarray2.html Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Have fun! -- Francesc Alted falted(a)pytables.org

1 0

PEP 328: Imports: Multi-Line and Absolute/Relative
by Aahz 08 Mar '04

08 Mar '04

Please see comp.lang.python (python-list) for discussion of the new PEP 328 that covers changes to the way ``import`` works. -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ "Do not taunt happy fun for loops. Do not change lists you are looping over." --Remco Gerlich, comp.lang.python

1 0

PyCon 2004 call for volunteers
by Itamar Shtull-Trauring 08 Mar '04

08 Mar '04

PyCon needs your help! We need volunteers to run the registration desk. Volunteers will help register conference goers, give out information about talk locations and so on. In addition to knowing they've helped make the conference a success, volunteers will get a black staff t-shirt. You can sign up at http://www.python.org/cgi-bin/moinmoin/PyconRegistrationDesk2004

1 0

ANNOUNCE: Rx4RDF and Rhizome 0.2.0
by Adam Souzis 07 Mar '04

07 Mar '04

Rx4RDF is application stack for building RDF-based applications and web sites. Rhizome is a Wiki-like content management and delivery system built on Rx4RDF that generalizes the wiki concept in several ways. What's new? RxPath has been completely reimplemented. Enhancements include: signficant performance boost, incrementally updates of the underlying model, atomic transactions and rollback, and RDF model independence through a simple API (includes adapters for 4Suite and Redland). Racoon is also signficantly faster as result of adding various caches throughout the request pipeline (taking advantage of the (nearly) side-effect free nature of XPath and XSLT). Rhizome now provides an optional staging/release workflow. In addition, there have several other enhancements, see http://rx4rdf.liminalzone.org/Rx4RDFChangelog for more details. More Info: * Rx4RDF provides a deterministic mapping between the RDF abstract syntax to the XPath data model, allowing you to query, transform and update a RDF model with languages syntactically indentical to XPath, XSLT and XUpdate (dubbed RxPath, RxSLT, and RxUpdate respectively). * Racoon is a simple application server that uses an RDF model for its data store, roughly analogous to RDF as Apache Cocoon is to XML. Racoon uses RxPath to translate arbitrary requests (currently HTTP, XML-RPC and command line arguments) to RDF resources, each of which can be associated with RxSLT and RxUpdate stylesheets. * Rhizome is a Wiki-like content management and delivery system built on Racoon that takes the concept of the Wiki to the next level: everything is editable, not just content but its meta-data and behavior, even the structure of the site itself. Furthermore, wiki entries are abstract globally unique RDF resources that can have any kind of content and whose presentation is contextual. Rhizome includes a couple of other stand-alone technologies that maybe of interest: * RhizML is a Wiki-like text formatting language that lets you write arbitrary XML or HTML, enabling you to author XML documents with (nearly) the same ease as a Wiki entry. * RxML is an alternative XML serialization for RDF that is designed for easy authoring in RhizML, allowing novices to author and edit RDF metadata. Homepage: http://rx4rdf.liminalzone.org/ Download: http://sourceforge.net/project/showfiles.php?group_id=85676 -- adam (asouzis @ user.sf.net)

1 0

ANN: PuLP, a Python LP (linear program) modeler
by Jean-Sebastien Roy 07 Mar '04

07 Mar '04

I would like to announce the release of PuLP v 1.0. PuLP is an LP modeler written in python. PuLP can generate MPS or LP files and call GLPK[1], COIN CLP/SBB[2], CPLEX[3] and XPRESS[4] to solve linear problems. PuLP provides a nice syntax for the creation of linear problems, and a simple way to call the solvers to perform the optimization. See the example below. You can get it at: http://www.jeannot.org/~js/code/pulp-1.0.tgz The latest version is always available at: http://www.jeannot.org/~js/code/index.en.html Multiple examples are provided. Thanks, Jean-Sebastien References: [1] http://www.gnu.org/software/glpk/glpk.html [2] http://www.coin-or.org/ [3] http://www.cplex.com/ [4] http://www.dashoptimization.com/ Example script: from pulp import * prob = LpProblem("test1", LpMinimize) # Variables x = LpVariable("x", 0, 4) y = LpVariable("y", -1, 1) z = LpVariable("z", 0) # Objective prob += x + 4*y + 9*z # Constraints prob += x+y <= 5 prob += x+z >= 10 prob += -y+z == 7 GLPK().solve(prob) # Solution for v in prob.variables(): print v.name, "=", v.varValue print "objective=", value(prob.objective)

1 0

ANN: Pyslide 0.4
by Ayose 07 Mar '04

07 Mar '04

Pyslide is a tiny program to make presentations. Presentations are created in an XML file, and showed using SDL (throught pygame). http://www.hispalinux.es/~setepo/pyslide/ Main Changes ============ * Execute external commands * Built-in lists * Timer and page-number bar * New effects style (more powerfull and easy) * Embedded images * More information on errors * Speed improvement * A lot of bugfixes Download ======== * http://www.hispalinux.es/~setepo/pyslide/download With subversion, $ svn co http://svn.hispalinux.es:8080/svn/pyslide/ You can use the Debian packages created by Tomas Guemes, available at <http://echeide.net/debian/pyslide/>. There is packages for i386 and PPC. -- Ayose Cazorla León Debian GNU/Linux - setepo

1 0

ANNOUNCE: Rx4RDF and Rhizome 0.2.0
by Adam Souzis 07 Mar '04

07 Mar '04

Rx4RDF is application stack for building RDF-based applications and web sites. Rhizome is a Wiki-like content management and delivery system built on Rx4RDF that generalizes the wiki concept in several ways. What's new? RxPath has been completely reimplemented. Enhancements include: signficant performance boost, incrementally updates of the underlying model, atomic transactions and rollback, and RDF model independence through a simple API (includes adapters for 4Suite and Redland). Racoon is also signficantly faster as result of adding various caches throughout the request pipeline (taking advantage of the (nearly) side-effect free nature of XPath and XSLT). Rhizome now provides an optional staging/release workflow. In addition, there have several other enhancements, see http://rx4rdf.liminalzone.org/Rx4RDFChangelog for more details. More Info: * Rx4RDF provides a deterministic mapping between the RDF abstract syntax to the XPath data model, allowing you to query, transform and update a RDF model with languages syntactically indentical to XPath, XSLT and XUpdate (dubbed RxPath, RxSLT, and RxUpdate respectively). * Racoon is a simple application server that uses an RDF model for its data store, roughly analogous to RDF as Apache Cocoon is to XML. Racoon uses RxPath to translate arbitrary requests (currently HTTP, XML-RPC and command line arguments) to RDF resources, each of which can be associated with RxSLT and RxUpdate stylesheets. * Rhizome is a Wiki-like content management and delivery system built on Racoon that takes the concept of the Wiki to the next level: everything is editable, not just content but its meta-data and behavior, even the structure of the site itself. Furthermore, wiki entries are abstract globally unique RDF resources that can have any kind of content and whose presentation is contextual. Rhizome includes a couple of other stand-alone technologies that maybe of interest: * RhizML is a Wiki-like text formatting language that lets you write arbitrary XML or HTML, enabling you to author XML documents with (nearly) the same ease as a Wiki entry. * RxML is an alternative XML serialization for RDF that is designed for easy authoring in RhizML, allowing novices to author and edit RDF metadata. Homepage: http://rx4rdf.liminalzone.org/ Download: http://sourceforge.net/project/showfiles.php?group_id=85676 -- adam (asouzis @ user.sf.net)

1 0

BayPIGgies: March 11, 7:30pm
by Aahz 06 Mar '04

06 Mar '04

The next meeting of BayPIGgies will be Thurs March 11 at 7:30pm. This meeting will have two sessions: * A new users experience of learning Python -- where the speaker asks the questions and the audience provides the answers. This session will be especially useful to other new Pythonistas. * An overview of a QA framework for enterprise applications BayPIGgies meetings are in Stanford, California. For more information and directions, see http://www.baypiggies.net/ -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ "Do not taunt happy fun for loops. Do not change lists you are looping over." --Remco Gerlich, comp.lang.python

1 0

Middleware 04: Call for Papers, Workshop and Tutorial proposals
by Middleware 04 06 Mar '04

06 Mar '04

Call for Papers: Middleware 2004 ACM/IFIP/USENIX International Middleware Conference (society sponsorship pending) Toronto, Ontario, Canada October 18th - 22nd, 2004 http://www.eecg.utoronto.ca/middleware2004/ Overview Requirements for faster development cycles, decreased development efforts, greater software reuse, and better end-to-end control over system resources are motivating the creation and use of middleware systems and middleware-based architectures. Middleware is systems software that resides between the applications and the underlying operating systems, network protocol stacks, and hardware. Its primary role is to functionally bridge the gap between application programs and the lower-level hardware and software infrastructure in order to coordinate how application components are connected and how they interoperate. Furthermore, middleware enables and simplifies the integration of components developed by multiple technology suppliers. In this sense middleware systems are sets of services and abstractions that facilitate the development and deployment of distributed applications in heterogeneous, distributed, computing environments. Next-generation distributed applications and systems will increasingly be developed using middleware. This dependency poses hard challenges, including latency hiding, masking partial failures, information assurance and security, legacy integration, dynamic service partitioning and load balancing, and end-to-end quality of service specification and enforcement. To address these challenges, researchers and practitioners need to discover and validate techniques, patterns, and optimizations for middleware frameworks, multi-level distributed resource management, and adaptive and reflective middleware architectures. Following the success of past conferences in this series, the 5th International Middleware Conference will be the premier event for middleware research and technology in 2004. The scope of the conference is the design, implementation, deployment, and evaluation of distributed system platforms and architectures for future computing and communication environments. Highlights of the conference will include a high quality technical program, tutorials, invited speakers, poster presentations, and workshops. The proceedings of Middleware 2004 will be published as a Springer-Verlag volume in the Lecture Notes in Computer Science Series. For paper formatting instructions see the Springer-Verlag guidelines for authors. All papers should be no more than 20 pages in length. For more detailed submission instructions, please visit the Middleware 2004 web site. Topics of Interest The topics of this conference include, but are not limited to: Distributed real-time and embedded middleware platforms Reliable and fault-tolerant middleware platforms Support for multimedia in middleware platforms Middleware for Grid computing Novel quality of service architectures and evaluation techniques Event-based, publish/subscribe and messaging-oriented middleware platforms Open architectures for reconfigurable middleware Adaptive and reflective middleware Aspect-oriented middleware Generative programming techniques for middleware development Middleware protocols and services for information assurance and security Formal methods and tools for reasoning about middleware systems and services Management and use of component-based systems in distributed environments Applications of middleware technologies, including telematics, command and control, avionics, and e-commerce Novel paradigms, APIs, and languages for distributed systems Integration of middleware with model-integrated computing architectures, such as the OMG's Model Driven Architecture (MDA) Extensions and refinements to RM-ODP, CORBA, J2EE, .NET, etc. Impact of emerging Internet technologies and standards on middleware platforms Integration of middleware platforms with Web services and Java technologies Distributed systems management and interactive configuration and development tools Issues of scalability in existing and new distributed systems platforms Engineering distributed systems in heterogeneous and mobile networks Middleware for ubiquitous and mobile computing Organization General Chair: Steve Vinoski (IONA Technologies, Inc.) Program Chair: Hans-Arno Jacobsen (University of Toronto, Canada) WiP Papers Chair: Jean Bacon (Cambridge University, UK) Tutorials Chair: Stefan Tai (IBM T.J. Watson, USA) Advanced Workshops Chair: Fabio Kon (USP, Brazil) Posters Chair: Eyal de Lara (University of Toronto, Canada) Local Arrangements Chair: Baochun Li (University of Toronto, Canada) Publicity Chair: Cristiana Amza (University of Toronto, Canada) Program Committee Gul Agha (U. of Illinois, Urbana Champaign, USA) Gustavo Alonso (ETH Zurich, Switzerland) Jean Bacon (Cambridge U., UK) Mark Baker (Canada) Guruduth Banavar (IBM T.J. Watson, USA) Alejandro Buchmann (Darmstadt U. of Technology, Germany) Andrew Campbell (Columbia U., USA) Roy Campbell (U. of Illinois, Urbana Champaign, USA) Harold Carr (Sun, USA) Geoff Coulson (Lancaster U., UK) Prem Devanbu (UC Davis, USA) Jan DeMeer (IHP-Microelectronics, Germany) Naranker Dulay (Imperial College, UK) Markus Endler (PUC-Rio, Brazil) Mike Feeley (U. of British Columbia, Canada) Chris Gill (Washington U., St. Louis, USA) Aniruddha Gokhale (Vanderbilt U., USA) Peter Honeyman (CITI, U. of Michigan, USA) Bettina Kemme (McGill U., Canada) Fabio Kon (U. of Sao Paulo, Brazil) Doug Lea (SUNY Oswego, USA)Joe Loyall (BBN Technologies, USA) Edmundo Madeira (U. of Campinas, Brazil) Keith Moore (HP Laboratories, USA) Hausi Muller (U. of Victoria, Canada) Klara Nahrstedt (U. of Illinois, Urbana Champaign, USA) Dennis Noll (Boeing, USA) Kerry Raymond (DSTC, Australia) Luis Rodrigues (U. of Lisboa, Portugal) Isabelle Rouvellou (IBM T.J. Watson, USA) Michael Stal (Siemens, Germany) Rick Schantz (BBN Technologies, USA) Douglas Schmidt (Vanderbilt U., USA) Jean-Bernard Stefani (INRIA, Grenoble, France) Joe Sventek (University of Glasgow, UK) Janos Sztipanovits (Vanderbilt U., USA) Stefan Tai (IBM T.J. Watson, USA) Peter Triantafillou (U. of Patras, Greece) Nalini Venkatasubramanian (U. of California, Irvine, USA) Werner Vogels (Cornell U., USA) Martina Zitterbart (U. of Karlsruhe, Germany) Submission Deadlines Abstract submission: March 30th, 2004 Research Papers: April 6th, 2004 Work in Progress Papers: April 6th, 2004 Posters: July 10th, 2004 Workshop Proposals: March 30th, 2004 Tutorial Proposals: May 11th, 2004 **All deadlines are 11:59pm PST.** Notification of acceptance (papers): Monday June 14th, 2004 Camera-ready papers due (papers): Monday July 12th, 2004 More Information For further information and submission instructions, please visit http://www.eecg.utoronto.ca/middleware2004/ . Please also see the following web sites of related conferences: - VLDB 2004 http://www.vldb04.org/ is the web site to the 30th International Conference on Very Large Data Bases also to be held in Toronto, Canada - PODC 2004 http://www.podc.org/podc2004/ is the web site for the 23rd Annual ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing) and of the Workshop on Concurrency and Synchronization in Java Programs which will be held in conjuction. We appologize if you receive multiple copies of this message.

1 0

A Byte of Python - Python Book / Tutorial
by python＠g2swaroop.net 05 Mar '04

05 Mar '04

Hello everyone, I have a written a book on Python which serves as a tutorial or guide for anyone who wants to learn Python. It specifically targets beginners but experienced programmers can benefit a lot with the special notes for them. The chapters are : Contents at a Glance 1. Introduction 2. Installing Python 3. First Steps 4. Basics 5. Operators and Expressions 6. Control Flow 7. Functions 8. Modules 9. Data Structures 10. Problem Solving - Writing a Python Script 11. Object-Oriented Programming 12. Input/Output 13. Exceptions 14. The Python Standard Library 15. More Python 16. What Next? 17. Appendices * Free/Libre and Open Source Software (FLOSS) * About The book is located at : http://www.python.g2swaroop.net/ During my first brush with Python, I felt there was a need for a good introductory guide to Python, and recent discussions on comp.lang.python reinforce that. I hope this book helps fill that void. This is my first attempt at writing anything so huge and so I am looking forward to your suggestions and criticisms so that I can further improve the book. Thanks, Swaroop C H www.python.g2swaroop.net

1 0