Python-Dev June 2005

python-dev@python.org

111 participants
115 discussions

by Skip Montanaro

I wrote PEP 304, "Controlling Generation of Bytecode Files": http://www.python.org/peps/pep-0304.html quite awhile ago. The first version appeared in January 2003 in response to questions from people about controlling/suppressing bytecode generation in certain situations. It sat idle for a long while, though from time-to-time people would ask about the functionality and I'd respond or update the PEP. In response to another recent question about this topic: http://mail.python.org/pipermail/python-list/2005-June/284775.html and a wave of recommendations by Raymond Hettinger regarding several other PEPs, I updated the patch to work with current CVS. Aside from one response by Thomas Heller noting that my patch upload failed (and which has been corrected since), I've seen no response either on python-dev or comp.lang.python. I really have no personal use for this functionality. I control all the computers on which I use Python and don't use any exotic hardware (which includes Windows as far with its multi-rooted file system as far as I'm concerned), don't run from read-only media or think that in-memory file systems are much of an advantage over OS caching. The best I will ever do with it is respond to people's inputs. I'd hate to see it sit for another two years. If someone out there is interested in this functionality and would benefit more from its incorporation into the core, I'd be happy to hand it off to you. So speak up folks, otherwise my recommendation is that it be put out of its misery. Skip

15 years, 9 months

refcounting vs PyModule_AddObject

by Michael Hudson

I've just fixed a bug where Py_INCREF wasn't called when it should have been before a call to PyModule_AddObject (rev. 2.62 of Modules/threadmodule.c). So I went looking for other instances of the same problem. I didn't find any (though I don't understand how _csv.c gets away with line 1579), but what I *did* find were absolutely masses of what seemed like unnecessary increfs. Consider this code from _hotshot.c: Py_INCREF(&LogReaderType); PyModule_AddObject(module, "LogReaderType", (PyObject *)&LogReaderType); Py_INCREF(&ProfilerType); PyModule_AddObject(module, "ProfilerType", (PyObject *)&ProfilerType); if (ProfilerError == NULL) ProfilerError = PyErr_NewException("hotshot.ProfilerError", NULL, NULL); if (ProfilerError != NULL) { Py_INCREF(ProfilerError); PyModule_AddObject(module, "ProfilerError", ProfilerError); } The first two calls are fine; it was an incref like this that was missing in threadmodule.c. The second seems like a reference "leak": PyErr_NewException returns a new reference, then the Py_INCREF/PyModule_AddObject pair is refcount neutral, so control falls off the end of the function owning a reference to ProfilerError. I think the Py_INCREF should just be removed, but I'm wondering if I'm missing something... Cheers, mwh -- MacOSX: Sort of like a pedigree persian cat. Very sleek, very sexy, but a little too prone to going cross-eyed, biting you on your thumb and then throwing up on your trousers. -- Jim's pedigree of operating systems, asr

15 years, 9 months

is type a usable feature?

by Paolino

Hello developers,I noticed my application was growing strangely while I was using type, then I tried this: while True: type('A',(),{}) and saw memory filling up.Is there a clean solution to that? I see it as a bug in python engeneering,that is why I wrote to you. Thanks a lot Paolino

15 years, 9 months

python-dev Summary for 2005-05-01 through 2005-05-16 [draft]

by Steven Bethard

Here's the May 01-15 draft. Sorry for the delay. Please check the Unicode summary at the end especially closely; I'm not entirely sure I got that one all right. Thanks! As always, please let us know if you have any corrections! Steve ====================== Summary Announcements ====================== ---------------------------------------------- PEP 340 Episode 2: Revenge of the With (Block) ---------------------------------------------- This fornight's Python-Dev was dominated again by another nearly 400 messages on the topic of anonymous block statements. The discussion was a little more focused than the last thanks mainly to Guido's introduction of `PEP 340`_. Discussion of this PEP resulted in a series of other PEPs, including * `PEP 342`_: Enhanced Iterators, which broke out into a separate PEP the parts of `PEP 340`_ that allowed code to pass values into iterators using ``continue EXPR`` and yield-expressions. * `PEP 343`_: Anonymous Block Redux, a dramatically simplified version of `PEP 340`_, which removed the looping nature of the anonymous blocks and the injection-of-exceptions semantics for generators. * `PEP 3XX`_: User Defined ("with") Statements, which proposed non-looping anonymous blocks accompanied by finalization semantics for iterators and generators in for loops. Various details of each of these proposals are discussed below in the sections: 1. `Enhanced Iterators`_ 2. `Separate APIs for Iterators and Anonymous Blocks`_ 3. `Looping Anonymous Blocks`_ 4. `Loop Finalization`_ At the time of this writing, it looked like the discussion was coming very close to a final agreement; `PEP 343`_ and `PEP 3XX`_ both agreed upon the same semantics for the block-statement, the keyword had been narrowed down to either ``do`` or ``with``, and Guido had agreed to add back in to `PEP 343`_ some form of exception-injection semantics for generators. .. _PEP 340: http://www.python.org/peps/pep-0340.html .. _PEP 342: http://www.python.org/peps/pep-0342.html .. _PEP 343: http://www.python.org/peps/pep-0343.html .. _PEP 3XX: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html [SJB] ========= Summaries ========= ------------------ Enhanced Iterators ------------------ `PEP 340`_ incorporated a variety of orthogonal features into a single proposal. To make the PEP somewhat less monolithic, the method for passing values into an iterator was broken off into `PEP 342`_. This method includes: * updating the iterator protocol to use .__next__() instead of .next() * introducing a new builtin next() * allowing continue-statements to pass values into iterators * allowing generators to receive values with a yield-expression Though these features had seemed mostly uncontroversial, Guido seemed inclined to wait for a little more motivation from the co-routiney people before accepting the proposal. Contributing threads: - `Breaking off Enhanced Iterators PEP from PEP 340 <http://mail.python.org/pipermail/python-dev/2005-May/053463.html>`__ [SJB] ------------------------------------------------ Separate APIs for Iterators and Anonymous Blocks ------------------------------------------------ `PEP 340`_ had originally proposed to treat the anonymous block protocol as an extension of the iterator protocol. Several problems with this approach were raised, including: * for-loops could accidentally be used with objects requiring blocks, meaning that resources would not get cleaned up properly * blocks could be used instead of for-loops, violating TOOWTDI As a result, both `PEP 343`_ and `PEP 3XX`_ propose decorators for generator functions that will wrap the generator object appropriately to match the anonymous block protocol. Generator objects without the proposed decorators would not be usable in anonymous block statements. Contributing threads: - `PEP 340 -- loose ends <http://mail.python.org/pipermail/python-dev/2005-May/053206.html>`__ - `PEP 340 -- concept clarification <http://mail.python.org/pipermail/python-dev/2005-May/053280.html>`__ [SJB] ------------------------ Looping Anonymous Blocks ------------------------ A few issues arose as a result of `PEP 340`_'s formulation of anonymous blocks as a variation on a loop. Because the anonymous blocks of `PEP 340`_ were defined in terms of while-loops, there was some discussion as to whether they should have an ``else`` clause like Python ``for`` and ``while`` loops do. There didn't seem to be one obvious interpretation of an ``else`` block though, so Guido rejected the ``else`` block proposal. The big issue with looping anonymous blocks, however, was in the handling of ``break`` and ``continue`` statements. Many use cases for anonymous blocks did not require loops. However, because `PEP 340`_ anonymous blocks were implemented in terms of loops, ``break`` and ``continue`` acted much like they would in a loop. This meant that in code like:: for item in items: with lock: if handle(item): break the ``break`` statement would only break out of the anonymous block (the ``with`` statement) instead of breaking out of the for-loop. This pretty much shot-down `PEP 340`_; there were too many cases where an anonymous block didn't look like a loop, and having it behave like one would have been a major stumbling block in learning the construct. As a result, both `PEP 343`_ and `PEP 3XX`_ were proposed as non-looping versions of `PEP 340`_. Contributing threads: - `PEP 340: Else clause for block statements <http://mail.python.org/pipermail/python-dev/2005-May/053190.html>`__ - `PEP 340 -- loose ends <http://mail.python.org/pipermail/python-dev/2005-May/053206.html>`__ - `PEP 340 -- concept clarification <http://mail.python.org/pipermail/python-dev/2005-May/053226.html>`__ - `PEP 340: Breaking out. <http://mail.python.org/pipermail/python-dev/2005-May/053223.html>`__ - `PEP 340: Non-looping version (aka PEP 310 redux) <http://mail.python.org/pipermail/python-dev/2005-May/053400.html>`__ - `PEP 340 - Remaining issues <http://mail.python.org/pipermail/python-dev/2005-May/053406.html>`__ - `PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340) <http://mail.python.org/pipermail/python-dev/2005-May/053503.html>`__ - `Merging PEP 310 and PEP 340-redux? <http://mail.python.org/pipermail/python-dev/2005-May/053591.html>`__ - `PEP 343 - Abstract Block Redux <http://mail.python.org/pipermail/python-dev/2005-May/053731.html>`__ [SJB] ----------------- Loop Finalization ----------------- Greg Ewing pointed out that a generator with a yield inside a block-statement would require additional work to guarantee its finalization. For example, if the generator:: def all_lines(filenames): for name in filenames: with open(name) as f: for line in f: yield line were used in code like:: for line in all_lines(filenames): if some_cond(line): break then unless the for-loop performed some sort of finalization on the all_lines generator, the last-opened file could remain open indefinitiely. As a result, `PEP 3XX`_ proposes that for-loops check for a __finish__() method on their iterators, and if one exists, call that method when the for-loop completes. Generators like all_lines above, that put a yield inside a block-statement, would then acquire a __finish__() method that would raise a TerminateIteration exception at the point of the last yield. The TerminateIteration exception would thus cause the block-statement to complete, guaranteeing that the generator was properly finalized. Contributing threads: - `PEP 340 - For loop cleanup, and feature separation <http://mail.python.org/pipermail/python-dev/2005-May/053432.html>`__ - `PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340) <http://mail.python.org/pipermail/python-dev/2005-May/053503.html>`__ - `PEP 343 - Abstract Block Redux <http://mail.python.org/pipermail/python-dev/2005-May/053731.html>`__ [SJB] ---------------------------- Breaking out of Nested Loops ---------------------------- As a result of some of the issues of looping anonymous blocks, a few threads discussed options for breaking out of nested loops. These mainly worked by augmenting the ``break`` statement with another keyword (or keywords) that would indicate which loop to break out of. One proposal suggested that ``break`` be followed with ``for`` or ``while`` to indicate which loop to break out of. But ``break for`` would only really be useful in a while-loop nested within a for-loop, and ``break while`` would only really be useful in a for-loop nested within a while-loop. That is, because loops could only be named by type, the proposal was only useful when loops of different types were mixed. This suggestion was thus discarded as not being general enough. A few other suggestions were briefly discussed: adding labels to loops, using an integer to indicate which "stack level" to break at, and pushing breaks onto a "break buffer", but Guido killed the discussion, saying, `"Stop all discussion of breaking out of multiple loops. It ain't gonna happen before my retirement." <http://mail.python.org/pipermail/python-dev/2005-May/053592.html>`__ Contributing threads: - `PEP 340: Breaking out. <http://mail.python.org/pipermail/python-dev/2005-May/053223.html>`__ - `PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340) <http://mail.python.org/pipermail/python-dev/2005-May/053503.html>`__ [SJB] ------------------------ The future of exceptions ------------------------ Ka-Ping Yee suggested that instead of passing (type, value, traceback) tuples in exceptions it would be better to put the traceback in value.traceback. Guido had also suggested this (in the `PEP 340`_ murk) but pointed out that this would not work as long as string exceptions exist (as there is nowhere to put the traceback). Guido noted that there are no concrete plans as to when string exceptions will be deprecated and removed (other than in 3.0 at the latest); he indicated that it could be sooner, if someone wrote a PEP with a timeline (e.g. deprecated in 2.5, gone in 2.6). Brett C. volunteered to write a PEP targetted at Python 3000 covering exception changes (base inheritance, standard attributes (e.g. .traceback), reworking the built-in exception inheritance hierarchy, and the future of bare except statements). Contributing threads: - `Tidier Exceptions <http://mail.python.org/pipermail/python-dev/2005-May/053671.html>`__ .. _PEP 340: http://www.python.org/peps/pep-0340.html [TAM] ----------------------------------- Unifying try/except and try/finally ----------------------------------- Reinhold Birkenfeld submitted a Pre-PEP to allow both except and finally clauses in try blocks. For example, a construction like:: try: <suite 1> except Ex1: <suite 2> <more except: clauses> else: <suite 3> finally: <suite 4> would be exactly the same as the legacy:: try: try: <suite 1> except Ex1: <suite 2> <more except: clauses> else: <suite 3> finally: <suite 4> Guido liked this idea (so much that he wanted to accept it immediately), and recommended that it was checked in as a PEP. However, Tim Peters pointed out that this functionality was removed from Python (by Guido) way back in 0.9.6, seemingly because there was confusion about exactly when the finally clause would be called (explicit is better than implicit!). Guido clarified that control would only pass forward, and indicated that he felt that since this is now available in Java (and C#) fewer people would be confused. The main concern about this change was that, while the cost was low, it seemed to add very little value. Contributing threads: - `Pre-PEP: Unifying try-except and try-finally <http://mail.python.org/pipermail/python-dev/2005-May/053290.html>`__ [TAM] ----------------- Decorator Library ----------------- Michele Simionato asked whether a module for commonly used decorators, or utilities to create decorators, was planned. Raymond Hettinger indicated that while this was likely in the long term, he felt that it was better if these first evolved via wikis, recipes, or mailing lists, so that a module would only be added once best practices and proven winners had emerged. In the meantime, there is both a `Decorator Library wiki page`_ and you can try out `Michele's library`_ [zip]. To assist with decorator creation, Michele would like a facility to copy a function. Phillip J. Eby noted that the informally-discussed proposal is to add a mutable __signature__ to functions to assist with signature preserving decorators. Raymond suggested a patch adding a __copy__ method to functions or a patch for the copy module, and Michele indicated that he would also like to subclass FunctionType with an user-defined __copy__ method. Contributing threads: - `my first post: asking about a "decorator" module <http://mail.python.org/pipermail/python-dev/2005-May/053316.html>`__ - `The decorator module <http://mail.python.org/pipermail/python-dev/2005-May/053446.html>`__ .. _Decorator Library wiki page: http://www.python.org/moin/PythonDecoratorLibrary .. _Michele's library: http://www.phyast.pitt.edu/~micheles/python/decorator.zip [TAM] --------------------- Hooking Py_FatalError --------------------- Errors that invoke Py_FatalError generally signify that the internal state of Python is in such a poor state that continuing (including raising an exception) is impossible or unwise; as a result, Py_FatalError outputs the error to stderr and calls abort(). m.u.k. would like to have a callback to hook Py_FatalError to avoid this call to abort(). The general consensus was that effort would be better directed to fixing the causes of fatal errors than hooking Py_FatalError. m.u.k.'s use case was for generating additional logging information; a `callback system patch`_ (revised by James William Pye) is available for those interested. Contributing threads: - `Need to hook Py_FatalError <http://mail.python.org/pipermail/python-dev/2005-May/053218.html>`__ .. _callback system patch: http://python.org/sf/1195571 ------------------- Chaining Exceptions ------------------- Ka-Ping Yee suggested adding information to exceptions when they are raised in the handler for another exception. For example:: def a(): try: raise AError except: raise BError raises an exception which is an instance of BError. This instance could have an attribute which is instance of AError, containing information about the original exception. Use cases include catching a low-level exception (e.g. socket.error) and turning it into a high-level exception (e.g. an HTTPRequestFailed exception) and handling problems in exception handling code. Guido liked the idea, and discussion fleshed out a tighter definition; however it was unclear whether adding this now was feasible - this would perhaps be best added in Python 3000. Contributing threads: - `Chained Exceptions <http://mail.python.org/pipermail/python-dev/2005-May/053672.html>`__ [TAM] ------------------------ Py_UNICODE Documentation ------------------------ Nicholas Bastin started a series of threads discussing an inconsistency between the Py_UNICODE docs and the behavior on some RedHat systems. The docs say that Py_UNICODE should be an alias for wchar_t when wchar_t is available and has 16 bits, but Nick found that pyconfig.h still reports PY_UNICODE_TYPE as wchar_t, even when PY_UNICODE_SIZE is 4. An extensive discussion between Nick, Marc-Andre Lemburg and Martin v. Löwis suggests that the possible Python-internal representations for Py_UNICODE are: * 4-byte wchar_t encoded as UTF-32 (UCS-4) * 2-byte wchar_t encoded as UTF-16 * unsigned short encoded as UTF-16 Python defaults to 2-byte mode, using wchar_t if available (and has 16 bits) and using unsigned short otherwise. You may end up with the 4-byte mode if TCL was built for UCS-4 (this overrides the defaults) or if you explicitly request it with --enable-unicode=ucs4. To get UCS-2 when TCL was built for UCS-4, you must explicitly request --enable-unicode=ucs2. Of course, this will mean that _tkinter can't be built anymore. Also noted by this discussion was that even with --enable-unicode=ucs2, Python continues to support surrogate pairs in the BMP. So for example, even with a UCS-2 build, u"\U00012345" encodes as a sequence of two characters; it does not produce a UnicodeError. At the time of this posting, it did not appear that there was a documentation patch available yet. Contributing threads: - `Py_UNICODE madness <http://mail.python.org/pipermail/python-dev/2005-May/053264.html>`__ - `New Py_UNICODE doc <http://mail.python.org/pipermail/python-dev/2005-May/053311.html>`__ - `Python's Unicode width default (New Py_UNICODE doc) <http://mail.python.org/pipermail/python-dev/2005-May/053574.html>`__ [SJB] =============== Skipped Threads =============== - `Keyword for block statements <http://mail.python.org/pipermail/python-dev/2005-May/053189.html>`__ - `PEP 340 - possible new name for block-statement <http://mail.python.org/pipermail/python-dev/2005-May/053195.html>`__ - `Generating nested data structures with blocks <http://mail.python.org/pipermail/python-dev/2005-May/053204.html>`__ - `PEP 340 -- Clayton's keyword? <http://mail.python.org/pipermail/python-dev/2005-May/053377.html>`__ - `PEP 340: Only for try/finally? <http://mail.python.org/pipermail/python-dev/2005-May/053258.html>`__ - `2 words keyword for block <http://mail.python.org/pipermail/python-dev/2005-May/053251.html>`__ - `anonymous blocks <http://mail.python.org/pipermail/python-dev/2005-May/053297.html>`__ - `"begin" as keyword for pep 340 <http://mail.python.org/pipermail/python-dev/2005-May/053315.html>`__ - `PEP 340: propose to get rid of 'as' keyword <http://mail.python.org/pipermail/python-dev/2005-May/053320.html>`__ - `PEP 340 keyword: after <http://mail.python.org/pipermail/python-dev/2005-May/053396.html>`__ - `PEP 340 keyword: Extended while syntax <http://mail.python.org/pipermail/python-dev/2005-May/053409.html>`__ - `PEP 340 - Remaining issues - keyword <http://mail.python.org/pipermail/python-dev/2005-May/053428.html>`__ - `PEP 340: Examples as class's. <http://mail.python.org/pipermail/python-dev/2005-May/053423.html>`__ - `Proposed alternative to __next__ and __exit__ <http://mail.python.org/pipermail/python-dev/2005-May/053514.html>`__ - `"with" use case: exception chaining <http://mail.python.org/pipermail/python-dev/2005-May/053665.html>`__ - `PEP 343: Resource Composition and Idempotent __exit__ <http://mail.python.org/pipermail/python-dev/2005-May/053767.html>`__ - `[Python-checkins] python/nondist/peps pep-0343.txt, 1.8, 1.9 <http://mail.python.org/pipermail/python-dev/2005-May/053766.html>`__ - `the current behavior of try: ... finally: <http://mail.python.org/pipermail/python-dev/2005-May/053692.html>`__ - `a patch to inspect and a non-feature request <http://mail.python.org/pipermail/python-dev/2005-May/053653.html>`__ - `Python 2.4 set objects and cyclic garbage <http://mail.python.org/pipermail/python-dev/2005-May/053630.html>`__ - `CHANGE BayPIGgies: May *THIRD* Thurs <http://mail.python.org/pipermail/python-dev/2005-May/053628.html>`__ - `Python continually calling sigprocmask() on FreeBSD 5 <http://mail.python.org/pipermail/python-dev/2005-May/053615.html>`__ - `Weekly Python Patch/Bug Summary <http://mail.python.org/pipermail/python-dev/2005-May/053213.html>`__ - `problems with memory management <http://mail.python.org/pipermail/python-dev/2005-May/053408.html>`__ - `Adding DBL_MANTISSA and such to Python <http://mail.python.org/pipermail/python-dev/2005-May/053372.html>`__ - `python-dev Summary for 2005-04-16 through 2005-04-30 [draft] <http://mail.python.org/pipermail/python-dev/2005-May/053383.html>`__ - `Python Language track at Europython, still possibilities to submit talks <http://mail.python.org/pipermail/python-dev/2005-May/053303.html>`__ - `(no subject) <http://mail.python.org/pipermail/python-dev/2005-May/053196.html>`__ - `Kernel panic writing to /dev/dsp with cmpci driver <http://mail.python.org/pipermail/python-dev/2005-May/053627.html>`__

15 years, 9 months

Re: [Python-Dev] Decimal floats as default (was: discussion about PEP239 and 240)

by Fredrik Johansson

On 6/22/05, Michael McLay <mclay(a)python.net> wrote: > This idea is dead on arrival. The change would break many applications and > modules. A successful proposal cannot break backwards compatibility. Adding a > dpython interpreter to the current code base is one possiblity. Is there actually much code around that relies on the particular precision of 32- or 64-bit binary floats for arithmetic, and ceases working when higher precision is available? Note that functions like struct.pack would be unaffected. If compatibility is a problem, this could still be a possibility for Python 3.0. In either case, compatibility can be ensured by allowing both n-digit decimal and hardware binary precision for floats, settable via a float context. Then the backwards compatible binary mode can be default, and "decimal mode" can be set with one line of code. d-suffixed literals create floats with decimal precision. There is the alternative of providing decimal literals by using separate decimal and binary float base types, but in my eyes this would be redundant. The primary use of binary floats is performance and compatibility, and both can be achieved with my proposal without sacrificing the simplicity and elegance of having a single type to represent non-integral numbers. It makes more sense to extend the float type with the power and versatility of the decimal module than to have a special type side by side with a default type that is less capable. Fredrik

15 years, 9 months

python-dev Summary for 2005-05-16 through 2005-05-31 [draft]

by Tony Meyer

You may have noticed that the summaries have been absent for the last month - apologies for that; Steve has been dutifully doing his part, but I've been caught up with other things. Anyway, Steve will post the May 01-15 draft shortly, and here's May 16-31. We should be able to get the first June one done fairly shortly, too. If anyone has time to flick over this and let me/Steve/Tim know if you have corrections, that would be great; thanks! =Tony.Meyer ============= Announcements ============= ---- QOTF ---- We have our first ever Quote of the Fortnight (QOTF), thanks to the wave of discussion over `PEP 343`_ and Jack Diederich: I still haven't gotten used to Guido's heart-attack inducing early enthusiasm for strange things followed later by a simple proclamation I like. Some day I'll learn that the sound of fingernails on the chalkboard is frequently followed by candy for the whole class. See, even threads about anonymous block statements can end happily! ;) .. _PEP 343: http://www.python.org/peps/pep-0343.html Contributing thread: - `PEP 343 - Abstract Block Redux <http://mail.python.org/pipermail/python-dev/2005-May/053828.html>`__ [SJB] ------------------ First PyPy Release ------------------ The first release of `PyPy`_, the Python implementation of Python, is finally available. The PyPy team has made impressive progress, and the current release of PyPy now passes around 90% of the Python language regression tests that do not depend deeply on C-extensions. The PyPy interpreter still runs on top of a CPython interpreter though, so it is still quite slow due to the double-interpretation penalty. .. _PyPy: http://codespeak.net/pypy Contributing thread: - `First PyPy (preview) release <http://mail.python.org/pipermail/python-dev/2005-May/053964.html>`__ [SJB] -------------------------------- Thesis: Type Inference in Python -------------------------------- Brett C. successfully defended his masters thesis `Localized Type Inference of Atomic Types in Python`_, which investigates some of the issues of applying type inference to the current Python language, as well as to the Python language augmented with type annotations. Congrats Brett! .. _Localized Type Inference of Atomic Types in Python: http://www.drifty.org/thesis.pdf Contributing thread: - `Localized Type Inference of Atomic Types in Python <http://mail.python.org/pipermail/python-dev/2005- May/053993.html>`__ [SJB] ========= Summaries ========= --------------------------- PEP 343 and With Statements --------------------------- The discussion on "anonymous block statements" brought itself closer to a real conclusion this fortnight, with the discussion around `PEP 343`_ and `PEP 3XX`_ converging not only on the semantics for "with statements", but also on semantics for using generators as with-statement templates. To aid in the adaptation of generators to with-statements, Guido proposed adding close() and throw() methods to generator objects, similar to the ones suggested by `PEP 325`_ and `PEP 288`_. The throw() method would cause an exception to be raised at the point where the generator is currently suspended, and the close() method would use throw() to signal the generator to clean itself up by raising a GeneratorExit exception. People seemed generally happy with this proposal and -- believe it or not -- we actually went an entire eight days without an email about anonymous block statements!! It looked as if an updated `PEP 343`_, including the new generator functionality, would be coming early the next month. So stay tuned. ;) .. _PEP 288: http://www.python.org/peps/pep-0288.html .. _PEP 325: http://www.python.org/peps/pep-0325.html .. _PEP 343: http://www.python.org/peps/pep-0343.html .. _PEP 3XX: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html Contributing threads: - `PEP 343 - Abstract Block Redux <http://mail.python.org/pipermail/python-dev/2005-May/053731.html>`__ - `Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux) <http://mail.python.org/pipermail/python -dev/2005-May/053812.html>`__ - `Example for PEP 343 <http://mail.python.org/pipermail/python-dev/2005-May/053855.html>`__ - `Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup <http://mail.python.org/pipermail/python-dev/2005-May/053885.html>`__ - `PEP 343 - New kind of yield statement? <http://mail.python.org/pipermail/python-dev/2005-May/053905.html>`__ - `PEP 342/343 status? <http://mail.python.org/pipermail/python-dev/2005-May/054007.html>`__ - `PEP 346: User defined statements (formerly known as PEP 3XX) <http://mail.python.org/pipermail/python-dev/2005- May/054014.html>`__ [SJB] ----------- Decimal FAQ ----------- Raymond Hettinger suggested that a decimal FAQ would shorten the module's learning curve, and drafted one. There were no objections, but few adjustments (to the list, at least). Raymond will probably make the FAQ available at some point. Contributing thread: - `Decimal FAQ <http://mail.python.org/pipermail/python-dev/2005-May/053982.html>`__ [TAM] --------------------- Constructing Decimals --------------------- A long discussion took place regarding whether the decimal constructor should or should not respect context settings, and whether matching the standard (and what the standard says) should be a priority. Raymond Hettinger took the lead in the status-quo (does not) corner, with Tim Peters leading the opposition. Tim and Guido eventually called in the standard's expert, Mike Cowlishaw. He gave a very thorough explanation of the history behind his decisions in this matter, and eventually weighed in on Raymond's side. As such, it seems that the status-quo has won (not that it was a competition, of course <wink>). For those that need to know, the unary plus operation, as strange as it looks, forces a rounding using the current context. As such, context-aware construction can be written:: val = +Decimal(string_repr) Contributing threads: - `Adventures with Decimal <http://mail.python.org/pipermail/python-dev/2005-May/053879.html>`__ - `Decimal construction <http://mail.python.org/pipermail/python-dev/2005-May/053882.html>`__ - `[Python-checkins] python/nondist/peps pep-0343.txt, 1.8, 1.9 <http://mail.python.org/pipermail/python-dev/2005- May/053766.html>`__ [TAM] ------------------------ Handling old bug reports ------------------------ Facundo Batista continued with his progress checking the open bug reports, looking for bugs that are specific to 2.2.1 or 2.2.2. The aim is to verify whether these bugs exist in current CVS, or are old-of-date. There are no longer any bugs in the 2.1.x or 2.2.x categories, and Facundo wondered whether removing those categories would be a good idea. The consensus was that there was no harm in leaving the categories there, but that changing the text to indicate that those versions are unmaintained would be a good idea. Raymond Hettinger reminded us that care needs to be taken in closing old bug reports. Particularly, a bug report should only be closed if (a) there are no means of reproducing the error, (b) it is impossible to tell what the poster meant, and they are no longer contactable, or (c) the bug is no longer present in current CVS. Contributing threads: - `Deprecating old bugs, now from 2.2.2 <http://mail.python.org/pipermail/python-dev/2005-May/054019.html>`__ - `Closing old bugs <http://mail.python.org/pipermail/python-dev/2005-May/054031.html>`__ - `Old Python version categories in Bug Tracker <http://mail.python.org/pipermail/python-dev/2005- May/054020.html>`__ [TAM] ------------------ Exception chaining ------------------ Ka-Ping Yee has submitted `PEP 344`_, which is a concrete proposal for exception chaining. It proposes three standard attributes on trackback objects: __context__ for implicit chaining, __cause__ for explicit chaining, and __traceback__ to point to the traceback. Guido likes the motivation and rationale, but feels that the specification needs more work. A lot of discussion about the specifics of the PEP took place, and Ka-Ping is working these into a revised version. One of the major questions was whether there is no need for both __context__ and __cause__ (to differentiate between explicit and implicit chaining). Guido didn't feel that there was, but others disagreed. Discussion branched off into whether which attributes should be double-underscored, or not. Guido's opinion is that it depends who "owns" the namespace, and with "magic" behaviour caused (or indicated) by the presence of the attribute. He felt that the underscores in the proposed exception attributes should remain. .. _PEP 344: http://www.python.org/peps/pep-0344.html Contributing threads: - `PEP 344: Exception Chaining and Embedded Tracebacks <http://mail.python.org/pipermail/python-dev/2005- May/053821.html>`__ - `PEP 344: Implicit Chaining Semantics <http://mail.python.org/pipermail/python-dev/2005-May/053944.html>`__ [Python-Dev] PEP 344: Implicit Chaining Semantics - `PEP 344: Explicit vs. Implicit Chaining <http://mail.python.org/pipermail/python-dev/2005-May/053945.html>`__ [Python-Dev] PEP 344: Explicit vs. Implicit Chaining - `Tidier Exceptions <http://mail.python.org/pipermail/python-dev/2005-May/053671.html>`__ [TAM] ------------------------------------ Adding content to exception messages ------------------------------------ Nicolas Fleury suggested that there should be a standard method of adding information to an existing exception (to re-raise it). Nick Coghlan suggested that this would be reasonably simple to do with PEP 344, if all exceptions were also new-style classes, but Nicolas indicated that this wouldn't work in some cases. Contributing threads: - `Adding content to exception messages <http://mail.python.org/pipermail/python-dev/2005-May/053903.html>`__ [Python-Dev] Adding content to exception messages [TAM] =============== Skipped Threads =============== - `Loading compiled modules under MSYS/MingGW? <http://mail.python.org/pipermail/python-dev/2005-May/053769.html>`__ - `RFC: rewrite fileinput module to use itertools. <http://mail.python.org/pipermail/python-dev/2005-May/053820.html>`__ - `Multiple interpreters not compatible with current thread module <http://mail.python.org/pipermail/python-dev/2005-May/053840.html>`__ - `Weekly Python Patch/Bug Summary <http://mail.python.org/pipermail/python-dev/2005-May/053875.html>`__ - `Request for dev permissions <http://mail.python.org/pipermail/python-dev/2005-May/053877.html>`__ - `python-dev Summary for 2005-05-01 through 2005-05-15 [draft] <http://mail.python.org/pipermail/python-dev/2005-May/053916.html>`__ - `AST manipulation and source code generation <http://mail.python.org/pipermail/python-dev/2005-May/053991.html>`__ - `Weekly Python Patch/Bug Summary <http://mail.python.org/pipermail/python-dev/2005-May/054000.html>`__ - `AST branch patches (was Re: PEP 342/343 status?) <http://mail.python.org/pipermail/python-dev/2005-May/054009.html>`__ - `[Python-checkins] python/dist/src/Lib/test test_site.py, 1.6, 1.7 <http://mail.python.org/pipermail/python-dev/2005-May/054011.html>`__ - `Split MIME headers into multiple lines near a space <http://mail.python.org/pipermail/python-dev/2005-May/054015.html>`__

15 years, 9 months

Explicitly declaring expected exceptions for a block

by Dmitry Dvoinikov

Excuse me if I couldn't find that in the existing PEPs, but wouldn't that be useful to have a construct that explicitly tells that we know an exception of specific type could happen within a block, like: ---------------------------------- ignore TypeError: do stuff [else: do other stuff] being essintially identical to try: do stuff except TypeError: pass [else: do other stuff] ---------------------------------- The reason for that being self-tests with lots and lots of little code snippets like this: try: c().foo() except TypeError: pass which could be written as ignore TypeError: c().foo() Sincerely, Dmitry Dvoinikov http://www.targeted.org/

15 years, 9 months

Decimal floats as default (was: discussion about PEP239 and 240)

by Fredrik Johansson

Hi all, raymond.hettinger at verizon.net Fri Jun 17 10:36:01 2005 wrote: > The future direction of the decimal module likely entails literals in > the form of 123.45d with binary floats continuing to have the form > 123.45. This conflicts with the rational literal proposal of having > 123.45 interpreted as 123 + 45/100. Decimal literals are a wonderful idea, especially if it means that decimals and floats can be made to interact with each other directly. But why not go one step further, making 123.45 decimal and 123.45b binary? In fact, I think a good case can be made for replacing the default float type with a decimal type. Decimal floats make life easier for humans accustomed to base 10, so they should be easy to use. This is particularly relevant given Python's relatively large user base of "non-programmers", but applies to many domains. Easy-to-use, correct rounding is essential in many applications that need to process human-readable data (round() would certainly be more meaningful if it operated on decimals). Not to mention that arbitrary precision arithmetic just makes the language more powerful. Rationals are inappropriate except in highly specialized applications because of the non-constant size and processing time, but decimals would only slow down programs by a (usually small) constant factor. I suspect most Python programs do not demand the performance hardware floats deliver, nor require the limited precision or particular behaviour of IEEE 754 binary floats (the need for machine-precision integers might be greater -- I've written "& 0xffffffffL" many times). Changing to decimal would not significantly affect users who really need good numeric performance either. The C interface would convert Python floats to C doubles as usual, and numarray would function accordingly. Additionally, "hardware" could be a special value for the precision in the decimal (future float) context. In that case, decimal floats could be phased in without breaking compatibility, by leaving hardware as the default precision. 123.45d is better than Decimal("123.45"), but appending "d" to specify a quantity with high precision is as illogical as appending "L" to an integer value to bypass the machine word size limit. I think the step from hardware floats to arbitrary-precision decimals would be as natural as going from short to unlimited-size integers. I've thought of the further implications for complex numbers and the math library, but I'll stop writing here to listen to feedback in case there is some obvious technical flaw or people just don't like the idea :-) Sorry if this has been discussed and/or rejected before (this is my first post to python-dev, though I've occasionally read the list since I started using Python extensively about two years ago). Fredrik Johansson

15 years, 9 months

Re: [Python-Dev] PEP for RFE 46738 (first draft)

by Skip Montanaro

Simon> XML is simply not suitable for database appplications, real time Simon> data capture and game/entertainment applications. I use XML-RPC as the communications protocol between an Apache web server and a middleware piece that talks to a MySQL database. The web server contains a mixture of CGI scripts written in Python and two websites written in Mason (Apache+mod_perl). Performance is fine. Give either of these a try: http://www.mojam.com/ http://www.musi-cal.com/ Simon> I'm sure other people have noticed this... or am I alone on this Simon> issue? :-) Probably not. XML-RPC is commonly thought of as slow, and if you operate with it in a completely naive fashion, I don't doubt that it can be. It doesn't have to be though. Do you have to be intelligent about caching frequently used information and returning results in reasonably sized chunks? Sure, but that's probably true of any database-backed application. Simon> Have a look at this contrived example: ... Simon> Which produces the output: >> pythonw -u "bench.py" Simon> Gherkin encode 0.120689361357 seconds Simon> Gherkin decode 0.395871262968 seconds Simon> XMLRPC encode 0.528666352847 seconds Simon> XMLRPC decode 9.01307819849 seconds That's fine, so XML-RPC is slower than Gherkin. I can't run the Gherkin code, but my XML-RPC numbers are a bit different than yours: XMLRPC encode 0.65 seconds XMLRPC decode 2.61 seconds That leads me to believe you're not using any sort of C XML decoder. (I mentioned sgmlop in my previous post. I'm sure /F has some other super-duper accelerator that's even faster.) I'm not saying that XML-RPC is the fastest thing on Earth. I'd be willing to bet it's a lot more interoperable than Gherkin is though: http://xmlrpc.scripting.com/directory/1568/implementations and probably will be for the forseeable future. Also, as you indicated, your example was a bit contrived. XML-RPC seems to be fast enough for many real-world applications. Here's a somewhat less-contrived example from the above websites: >>> orca <ServerProxy for orca.mojam.com:5007/RPC2> >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 200, 50, 10000) ; print time.time() - t 1.28429102898 >>> len(x[1]) 200 >>> x[1][0] ['MusicEntry', {'venue': 'Jazz Showcase', 'address': '', 'price': '', 'keywords': ['.jz.1369'], 'event': '', 'city': 'Chicago', 'end': datetime.datetime(2005, 6, 19, 0, 0), 'zip': '', 'start': datetime.datetime(2005, 6, 14, 0, 0), 'state': 'IL', 'program': '', 'email': 'skip(a)mojam.com', 'info': '', 'update_time': datetime.datetime(2005, 6, 8, 0, 0), 'address1': '59 West Grand Avenue', 'address2': '', 'address3': '', 'venueid': 2630, 'key': 816875, 'submit_time': datetime.datetime(2005, 6, 8, 0, 0), 'active': 1, 'merchandise': '', 'tickets': '', 'name': '', 'addressid': 17056, 'performers': ['Cedar Walton Quartet'], 'country': '', 'venueurl': '', 'time': '', 'performerids': [174058]}] >>> orca.checkpoint() 'okay' >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 200, 50, 10000) ; print time.time() - t 1.91681599617 orca is an xmlrpclib proxy to the aforementioned middleware component. (These calls are being made to a production server.) The search() call gets the first 200 concert listings within a 50-mile radius of Chicago. x[1][0] is the first item returned. All 200 returned items are of the same complexity. 1.28 seconds certainly isn't Earth-shattering performance. Even worse is the 1.92 seconds after the checkpoint (which flushes the caches forcing the MySQL database to be queried for everything). Returning 200 items is also contrived. Real users can't ask for that many items at a time through the web interface. Cutting it down to 20 items (which is what users get by default) shows a different story: >>> orca.checkpoint() 'okay' >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 20, 50, 10000) ; print time.time() - t 0.29478096962 >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 20, 50, 10000) ; print time.time() - t 0.0978591442108 The first query after a checkpoint is slow because we have to go to MySQL a few times, but once everything's cached, things are fast. The 0.1 second time for the last call is going to be almost all XML-RPC overhead, because all the data's already been cached. I find that acceptable. If you go to the Mojam website and click "Chicago", the above query is pretty much what's performed (several other queries are also run to get corollary info). I still find it hard to believe that yet another serialization protocol is necessary. XML is certainly overkill for almost everything. I'll be the first to admit that. (I have to struggle with it as a configuration file format at work.) However, it is certainly widely available. Skip

15 years, 9 months

subprocess.call() and stdin

by Stuart Bishop

Redirecting to python-dev for discussion. When I invoke subprocess.call(), I often want to ensure that the subprocess' stdin is closed. This ensures it will die if the subprocess attempts to read from stdin rather than block. This could be done if the subprocess.call() helper closes the input if stdin=subprocess.PIPE, which may be the only sane way to handle this argument (I can't think of any use cases for spawning a subprocess with an input stream that nothing can write too). It could also be done by adding a subprocess.CLOSED constant, which if passed to Popen causes a new closed file descriptor to be given to the subprocess. -------- Original Message -------- Bugs item #1220113, was opened at 2005-06-14 15:04 Message generated for change (Comment added) made by zenzen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1220113&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Resolution: Rejected Priority: 5 Submitted By: Stuart Bishop (zenzen) Assigned to: Peter Åstrand (astrand) Summary: subprocess call() helper should close stdin if PIPE Initial Comment: The following code snippet should die instead of hang. >>> from subprocess import call, PIPE >>> rv = call(['/usr/bin/bzip2', '-c'], stdout=PIPE, stdin=PIPE) It makes no sense not to close stdin if it is PIPE because the stream cannot be accessed. The use case for this is ensuring a subprocess that detects if it is connected to a terminal or not runs in 'batch' mode, and that it will die instead of hang if it unexpectidly attempts to read from stdin. Workaround is to use Popen instead. ---------------------------------------------------------------------- >Comment By: Stuart Bishop (zenzen) Date: 2005-06-22 16:12 Message: Logged In: YES user_id=46639 I can't think of any uses cases for wanting to create an inaccessible pipe and give it to the child. Wanting to pass a closed file handle is common. It is needed when calling a program that behaves differently if its stdin is a terminal or not. Or when you simply would prefer the subprocess to die if it attempts to read from its stdin rather than block. Using Popen instead of call is s simpler workaround than creating and closing a file descriptor and passing it in. Perhaps what is needed is a new constant, subprocess.CLOSED which creates a new file descriptor and closes it? This would be useful for Popen too, allowing call() to remain a think and trivially documented wrapper? ---------------------------------------------------------------------- Comment By: Peter Åstrand (astrand) Date: 2005-06-22 02:08 Message: Logged In: YES user_id=344921 >It makes no sense not to close stdin if it is PIPE >because the stream cannot be accessed True, but what if you actually *want* to create an inaccessible pipe, and give it to the child? Currently, the call() wrapper is *very* short and simple. I think this is very good. For example, it allows us to document it in two sentences. You get what you ask for: If you use call() with strange arguments, you'll get a somewhat strange behavíour. I see no point in introducing lots of sanity checks in the wrapper functions. >The use case for this is ensuring a subprocess that >detects if it is connected to a terminal or not runs in >batch' mode, and that it will die instead of hang if >it unexpectidly attempts to read from stdin I'm not sure I understand what you want, but if you want to have stdin connected to a closed file descriptor, just pass one: >>> from subprocess import call, PIPE >>> rv = call(['/usr/bin/bzip2', '-c'], stdout=PIPE, stdin=4711) (Of course, you should verify that 4711 is unused.) If you don't agree with me, post to python-dev for discussion. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1220113&group_… -- Stuart Bishop <stuart(a)stuartbishop.net> http://www.stuartbishop.net/

15 years, 9 months

← Newer
1
2
3
4
5
6
7
...
12
Older →

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev June 2005