Mailman 3 March 2007 - Python-Dev

Re: [Python-Dev] An updated extended buffer PEP
by Travis Oliphant 29 Mar '07

29 Mar '07

Carl Banks wrote: > Travis E. Oliphant wrote: >> I think we are getting closer. What do you think about Greg's idea >> of basically making the provider the bufferinfo structure and having >> the exporter handle copying memory over for shape and strides if it >> wants to be able to change those before the lock is released. > > It seems like it's just a different way to return the data. You could > do it by setting values through pointers, or do it by returning a > structure. Which way you choose is a minor detail in my opinion. I'd > probably favor returning the information in a structure. > > I would consider adding two fields to the structure: > > size_t structsize; /* size of the structure */ Why is this necessary? can't you get that by sizeof(bufferinfo)? > PyObject* releaser; /* the object you need to call releasebuffer on */ Is this so that another object could be used to manage releases if desired? -Travis

3 4

Has anyone been in touch with Fred Drake?
by skip＠pobox.com 29 Mar '07

29 Mar '07

I've been seeing bounces for Fred Drake's Comcast email for a couple days. Fred, are you listening? If not, does someone else have a non-Comcast email link to Fred? (I assume his acm.org address is just an alias which might well point to his Comcast mailbox.) Thx, Skip

3 2

An updated extended buffer PEP
by Travis Oliphant 28 Mar '07

28 Mar '07

Hi Carl and Greg, Here is my updated PEP which incorporates several parts of the discussions we have been having. -Travis

4 8

[ 1688393 ] sock.recvfrom(-24) crashes
by Facundo Batista 28 Mar '07

28 Mar '07

I applied the patch in this bug to the trunk. As it's a bug, and a very nasty one (it causes an ugly crash), please consider backporting it to 2.5.x. If you apply this to 2.5.x, just close the bug. Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

1 0

PyPy 1.0: JIT compilers for free and more
by Armin Rigo 28 Mar '07

28 Mar '07

Hi all, Sorry for the spamming. I hope this will be of interest to some of you. Armin ========================================== PyPy 1.0: JIT compilers for free and more ========================================== Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! Although still not mature enough for general use, PyPy 1.0 materializes for the first time the full extent of our original vision: - A flexible Python interpreter, written in "RPython": - Mostly unaware of threading, memory and lower-level target platform aspects. - Showcasing advanced interpreter features and prototypes. - Passing core CPython regression tests, translatable to C, LLVM and .NET. - An advanced framework to translate such interpreters and programs: - That performs whole type-inference on RPython programs. - Can weave in threading, memory and target platform aspects. - Has low level (C, LLVM) and high level (CLI, Java, JavaScript) backends. - A **Just-In-Time Compiler generator** able to **automatically** enhance the low level versions of our Python interpreter, leading to run-time machine code that runs algorithmic examples at speeds typical of JITs! Previous releases, particularly the 0.99.0 release from February, already highlighted features of our Python implementation and the abilities of our translation approach but the **new JIT generator** clearly marks a major research result and gives weight to our vision that one can generate efficient interpreter implementations, starting from a description in a high level language. We have prepared several entry points to help you get started: * The main entry point for JIT documentation and status: http://codespeak.net/pypy/dist/pypy/doc/jit.html * The main documentation and getting-started PyPy entry point: http://codespeak.net/pypy/dist/pypy/doc/index.html * Our online "play1" demos showcasing various Python interpreters, features (and a new way to program AJAX applications): http://play1.codespeak.net/ * Our detailed and in-depth Reports about various aspects of the project: http://codespeak.net/pypy/dist/pypy/doc/index-report.html In the next few months we are going to discuss the goals and form of the next stage of development - now more than ever depending on your feedback and contributions - and we hope you appreciate PyPy 1.0 as an interesting basis for greater things to come, as much as we do ourselves! have fun, the PyPy release team, Samuele Pedroni, Armin Rigo, Holger Krekel, Michael Hudson, Carl Friedrich Bolz, Antonio Cuni, Anders Chrigstroem, Guido Wesdorp Maciej Fijalkowski, Alexandre Fayolle and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html What is PyPy? ================================ Technically, PyPy is both a Python interpreter implementation and an advanced compiler, or more precisely a framework for implementing dynamic languages and generating virtual machines for them. The framework allows for alternative frontends and for alternative backends, currently C, LLVM and .NET. For our main target "C", we can can "mix in" different garbage collectors and threading models, including micro-threads aka "Stackless". The inherent complexity that arises from this ambitious approach is mostly kept away from the Python interpreter implementation, our main frontend. PyPy is now also a Just-In-Time compiler generator. The translation framework contains the now-integrated JIT generation technology. This depends only on a few hints added to the interpreter source and should be able to cope with the changes to the interpreter and be generally applicable to other interpreters written using the framework. Socially, PyPy is a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPy would not have gotten as far as it has without the coding, feedback and general support from numerous people. Formally, many of the current developers were involved in executing an EU contract with the goal of exploring and researching new approaches to language and compiler development and software engineering. This contract's duration is about to end this month (March 2007) and we are working and preparing the according final review which is scheduled for May 2007. For the future, we are in the process of setting up structures to help maintain conceptual integrity of the project and to discuss and deal with funding opportunities related to further PyPy sprinting and developments. See here for results of the discussion so far: http://codespeak.net/pipermail/pypy-dev/2007q1/003577.html 1.0.0 Feature highlights ============================== Here is a summary list of key features included in PyPy 1.0: - The Just-In-Time compiler generator, now capable of generating the first JIT compiler versions of our Python interpreter: http://codespeak.net/pypy/dist/pypy/doc/jit.html - More Python interpreter optimizations (a CALL_METHOD bytecode, a method cache, rope-based strings), now running benchmarks at around half of CPython's speed (without the JIT): http://codespeak.net/pypy/dist/pypy/doc/interpreter-optimizations.html - The Python interpreter can be translated to .NET and enables interactions with the CLR libraries: http://codespeak.net/pypy/dist/pypy/doc/cli-backend.html http://codespeak.net/pypy/dist/pypy/doc/clr-module.html - Aspect Oriented Programming facilities (based on mutating the Abstract Syntax Tree): http://codespeak.net/pypy/dist/pypy/doc/aspect_oriented_programming.html http://codespeak.net/pypy/extradoc/eu-report/D10.1_Aspect_Oriented_Programm… - The JavaScript backend has evolved to a point where it can be used to write AJAX web applications with it. This is still an experimental technique, though. For demo applications which also showcase various generated Python and PROLOG interpreters, see: http://play1.codespeak.net/ - Proxying object spaces and features of our Python interpreter: - Tainting: a 270-line proxy object space tracking and boxing sensitive information within an application. - Transparent proxies: allow the customization of both application and builtin objects from application level code. Now featuring an initial support module (tputil.py) for working with transparent proxies. For a detailed description and discussion of high level backends and Python interpreter features, please see our extensive "D12" report: http://codespeak.net/pypy/extradoc/eu-report/D12.1_H-L-Backends_and_Feature… Funding partners and organisations ===================================================== PyPy development and activities happen as an open source project and with the support of a consortium partially funded by a 28 month European Union IST research grant for the period from December 2004 to March 2007. The full partners of that consortium are: Heinrich-Heine University (Germany), Open End (Sweden) merlinux GmbH (Germany), tismerysoft GmbH (Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden), Impara (Germany)

3 2

Standard Image and Sound classes (was: SoC proposal: multimedia library)
by Lino Mastrodomenico 28 Mar '07

28 Mar '07

2007/3/26, Pete Shinners <pete(a)shinners.org>: > My main question is what is the image and sound container passed back to Python? > This single thing along would be worth a SoC if it could be implemented across > all libraries. > > Will your image objects be transferrable to PIL, Pygame, PyOpengl, Numpy, > Pythonmagick, Pycairo, wxPython, etc? It would be best if this could avoid the > "fromstring/tostring" mojo that always requires and extra copy of the data for > each transfer. I agree. I withdrew my original "multimedia library" idea and submitted a proposal for the creation of two standard Image and Sound classes. If the PSF accepts my proposal and if Google grants a student slot for me, I will try to implement an interface for the two classes that can work well for everyone and be easily usable from both the Python and the C side. I guess this will require a PEP, if this is the case I will volunteer to write it. And, most important, I will create patches for the appropriate standard library modules and for a lot of external libraries, to try to make the use of the two new classes really interoperable. Finally, as Greg suggested, adding the new buffer interface to the two classes can be a really good idea. I'm following with interest that discussion. Keeping fingers crossed... -- Lino Mastrodomenico E-mail: l.mastrodomenico(a)gmail.com

2 3

Re: [Python-Dev] [Fwd: PEP 0305 (small problem with the CSV reader)]
by Andrew McNamara 28 Mar '07

28 Mar '07

>First of all, let me say thank you for the CSV module. Thanks. >I've been using it and today is the first time I hit a minor bump in the road. >What happened is I opened this file with genome annotations with a >long field and the error "field larger than field limit" showed up. >From what I can see it is in the "static int parse_add_char(ReaderObj >*self, char c)" function. >This function uses the static long field_limit = 128 * 1024; /* max >parsed field size */ >I'm not sure if this is supposed to be recomputed or if there is >something I need to do to change it, but for right now it just says my >row is bigger than 131,072 and stops. >I don't think Python 2.5 has any such string length limitations and >this shouldn't be. This limit was added back in January 2005 to provide some protection against the situation where the parser is returning fields directly from a file, and the file contains a mismatched quote character: this would otherwise result in the entire file being unexpectedly read into memory. You can change the limit with the csv.field_size_limit() method. As you note, it defaults to 128K, but you can set it to anything up to (2**31)-1 or 2147483647 (about 2 billion). BTW, I've taken the liberty of CC'ing this to the python-dev list, so the motivation for this feature is recorded - it caused me some head scratching, and I added it. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/

1 0

Re: [Python-Dev] An updated extended buffer PEP
by Travis Oliphant 28 Mar '07

28 Mar '07

Lisandro Dalcin wrote: > On 3/26/07, Travis Oliphant <oliphant.travis(a)ieee.org> wrote: >> Here is my updated PEP which incorporates several parts of the >> discussions we have been having. > > Travis, it looks really good, below my comments I hope you don't mind me replying to python-dev. > > 1- Is it hard to EXTEND PyBufferProcs in order to be able to use all > this machinery in Py 2.X series, not having to wait until Py3k? No, I don't think it will be hard. I just wanted to focus on Py3k since it is going to happen before Python 2.6 and I wanted it discussed in that world. > > 2- Its not clear for me if this PEP will enable object types defined > in the Python side to export buffer info. This is a feature I really > like in numpy, and simplifies my life a lot when I need to export > memory for C/C++ object wrapped with the help of tools like SWIG. This PEP does not address that. You will have to rely on the objects themselves for any such information. > > 3- Why not to constraint the returned 'view' object to be of a > specific type defined in the C side (and perhaps available in the > Python side)? This 'view' object could maintain a reference to the > base object containing the data, could call releasebuffer using the > base object when the view object is decref'ed, and can have a flag > field for think like OWN_MEMORY, OWN_SHAPE, etc in order to properly > manage memory deallocation. Does all this make sense? Yes, that was my original thinking and we are kind of coming back to it after several iterations. Perhaps, though we can stick with an object-less buffer interface but have this "view object" as an expanded buffer object. -Travis

2 1

Summary of Tracker Issues
by Tracker 27 Mar '07

27 Mar '07

ACTIVITY SUMMARY (03/20/07 - 03/27/07) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1646 open ( +2) / 8583 closed ( +1) / 10229 total ( +3) Average duration of open issues: 747 days. Median duration of open issues: 695 days. Open Issues Breakdown open 1646 ( +2) pending 0 ( +0) Issues Created Or Reopened (3) ______________________________ Buy Tramadol online 03/25/07 http://bugs.python.org/issue1023 created conor2k Testing bugs.python.org domain 03/26/07 CLOSED http://bugs.python.org/issue1024 created izak Test issue 03/26/07 http://bugs.python.org/issue1025 created loewis Issues Now Closed (2) _____________________ Personal loans. Bad credit personal loans. Unsecured personal l 1 days http://bugs.python.org/issue1002 dubois Testing bugs.python.org domain 0 days http://bugs.python.org/issue1024 izak

1 0

SoC proposal: multimedia library
by Lino Mastrodomenico 27 Mar '07

27 Mar '07

Hello everyone, I would like to participate as a student in google Summer of Code and I'm interested in feedback on a multimedia library for Python. The library I propose should have the following features: * the capability to extract and decompress video and audio from a few common multimedia file format; * and, vice versa, it can create a compressed video file from a series of still images and from uncompressed audio; * it should have an interface as simple and pythonic as possible; * it must be cross-platform, easy to extend and not be limited to a single file container format or to some specific audio or video compression format; * at least a subset of the supported formats should be available without any external dependency. Why? I think this can be an useful addition to the standard library: the goal isn't to reimplement a full fledged media player like mplayer or VLC, but to offer, e.g., a very simple way to add multimedia animations to a Tkinter application. Or, for an application that generates animations, a simple way to save them in a common file format that can be played by existing media players. How? I know that this may sound like a project way too big for SoC, but I think it may be feasible: I plan to split it in two parts. A low level part responsible for splitting encoded audio and video data from multimedia files (demuxing) and putting it together in new multimedia files (muxing). I plan to initially only support AVI and MOV/Quicktime files because they are relatively simple and well documented (and I have already written Python programs for manipulating these two formats, so I know them pretty well). New container formats can be added later and, if the library interface meets the requirements, existing application will be able to use them without any modification. The second part of the library is a collection of codecs for compressing and decompressing audio and video streams. I don't plan to invest too much time here: the idea is to write only a few ctypes wrappers around existing libraries for common formats (e.g. MPEG4/XVID and Theora for video and MP3 and Vorbis for audio) and/or use existing third-part Python bindings. Again, new codecs can be added later. Similar libraries. There are already a number of multimedia libraries both inside and outside of the standard library, but AFAIK they all have limitations: * a lot of them are specific for a single container format or codec (e.g. aifc, wave, oggpy, python-vorbis...); * they aren't cross-platform (e.g. videoreader). Is there any interest in a library of this kind (inside or outside of the stdlib)? Suggestions? Criticism? -- Lino Mastrodomenico E-mail: l.mastrodomenico(a)gmail.com

7 7