Python-Dev August 2004

python-dev@python.org

170 participants
281 discussions

RE: [Python-Dev] Those import related syntax errors again...

by Samuele Pedroni

Hi. [Mark Hammond] > The point isn't about my suffering as such. The point is more that > python-dev owns a tiny amount of the code out there, and I don't believe we > should put Python's users through this. > > Sure - I would be happy to "upgrade" all the win32all code, no problem. I > am also happy to live in the bleeding edge and take some pain that will > cause. > > The issue is simply the user base, and giving Python a reputation of not > being able to painlessly upgrade even dot revisions. I agree with all this. [As I imagined explicit syntax did not catch up and would require lot of discussions.] [GvR] > > Another way is to use special rules > > (similar to those for class defs), e.g. having > > > > <frag> > > y=3 > > def f(): > > exec "y=2" > > def g(): > > return y > > return g() > > > > print f() > > </frag> > > > > # print 3. > > > > Is that confusing for users? maybe they will more naturally expect 2 > > as outcome (given nested scopes). > > This seems the best compromise to me. It will lead to the least > broken code, because this is the behavior that we had before nested > scopes! It is also quite easy to implement given the current > implementation, I believe. > > Maybe we could introduce a warning rather than an error for this > situation though, because even if this behavior is clearly documented, > it will still be confusing to some, so it is better if we outlaw it in > some future version. > Yes this can be easy to implement but more confusing situations can arise: <frag> y=3 def f(): y=9 exec "y=2" def g(): return y return y,g() print f() </frag> What should this print? the situation leads not to a canonical solution as class def scopes. or <frag> def f(): from foo import * def g(): return y return g() print f() </frag> [Mark Hammond] > > This probably won't be a very popular suggestion, but how about pulling > > nested scopes (I assume they are at the root of the problem) > > until this can be solved cleanly? > > Agreed. While I think nested scopes are kinda cool, I have lived without > them, and really without missing them, for years. At the moment the cure > appears worse then the symptoms in at least a few cases. If nothing else, > it compromises the elegant simplicity of Python that drew me here in the > first place! > > Assuming that people really _do_ want this feature, IMO the bar should be > raised so there are _zero_ backward compatibility issues. I don't say anything about pulling nested scopes (I don't think my opinion can change things in this respect) but I should insist that without explicit syntax IMO raising the bar has a too high impl cost (both performance and complexity) or creates confusion. [Andrew Kuchling] > >Assuming that people really _do_ want this feature, IMO the bar should be > >raised so there are _zero_ backward compatibility issues. > > Even at the cost of additional implementation complexity? At the cost > of having to learn "scopes are nested, unless you do these two things > in which case they're not"? > > Let's not waffle. If nested scopes are worth doing, they're worth > breaking code. Either leave exec and from..import illegal, or back > out nested scopes, or think of some better solution, but let's not > introduce complicated backward compatibility hacks. IMO breaking code would be ok if we issue warnings today and implement nested scopes issuing errors tomorrow. But this is simply a statement about principles and raised impression. IMO import * in an inner scope should end up being an error, not sure about 'exec's. We will need a final BDFL statement. regards, Samuele Pedroni.

15 years, 11 months

mimetypes and _winreg

by Mike Brown

I thought it would be nice to try to improve the mimetypes module by having it, on Windows, query the Registry to get the mapping of filename extensions to media types, since the mimetypes code currently just blindly checks posix-specific paths for httpd-style mapping files. However, it seems that the way to get mappings from the Windows registry is excessively slow in Python. I'm told that the reason has to do with the limited subset of APIs that are exposed in the _winreg module. I think it is that EnumKey(key, index) is querying for the entire list of subkeys for the given key every time you call it. Or something. Whatever the situation is, the code I tried below is way slower than I think it ought to be. Does anyone have any suggestions (besides "write it in C")? Could _winreg possibly be improved to provide an iterator or better interface to get the subkeys? (or certain ones? There are a lot of keys under HKEY_CLASSES_ROOT, and I only need the ones that start with a period). Should I file this as a feature request? Thanks -Mike from _winreg import HKEY_CLASSES_ROOT, OpenKey, EnumKey, QueryValueEx i = 0 typemap = {} try: while 1: subkeyname = EnumKey(HKEY_CLASSES_ROOT, i) try: subkey = OpenKey(HKEY_CLASSES_ROOT, subkeyname) if subkeyname[:1] == '.': data = QueryValueEx(subkey, 'Content Type')[0] print subkeyname, '=', data typemap[subkeyname] = data # data will be unicode except EnvironmentError, WindowsError: pass i += 1 except WindowsError: pass

15 years, 11 months

Another test_compiler mystery

by Tim Peters

I've noticed several times now, in both debug and release builds, that if I run regrtest.py with -uall, *sometimes* it just stops after running test_compiler: $ python_d regrtest.py -uall test_grammar test_opcodes ... test_compare test_compile test_compiler $ There's no indication of error, it just ends. It's not consistent. Happened once when I was running with -v, and test_compiler's output ended here: ... compiling C:\Code\python\lib\test\test_operator.py compiling C:\Code\python\lib\test\test_optparse.py compiling C:\Code\python\lib\test\test_os.py compiling C:\Code\python\lib\test\test_ossaudiodev.py compiling C:\Code\python\lib\test\test_parser.py In particular, there's no Ran M tests in Ns output, so it doesn't look like unittest (let alone regrtest) ever got control back. Hmm. os.listdir() is in sorted order on NTFS, so test_compiler should be chewing over a lot more files after test_parser.py. *This* I could blame on a blown C stack -- although I'd expect a much nastier symptom then than just premature termination. Anyone else?

16 years, 3 months

Alternative Implementation for PEP 292: Simple String Substitutions

by Raymond Hettinger

Before it's too late and the API gets frozen, I would like to propose an alternate implementation for PEP292 that exposes two functions instead of two classes. Current way: print Template('Turn $direction') % dict(direction='right') Proposed: print dollarsub('Turn $direction', dict(direction='right')) or: print dollarsub('Turn $direction', direction='right') My main issue with the current implementation is that we get no leverage from using a class instead of a function. Though the API is fairly simple either way, it is easier to learn and document functions than classes. We gain nothing from instantiation -- the underlying data remains immutable and no additional state is attached. The only new behavior is the ability to apply the mod operator. Why not do it in one step. I had thought a possible advantage of classes was that they could be usefully subclassed. However, a little dickering around showed that to do anything remotely interesting (beyond changing the pattern alphabet) you have to rewrite everything by overriding both the method and the pattern. Subclassing gained you nothing, but added a little bit of complexity. A couple of simple exercises show this clearly: write a subclass using a different escape character or one using dotted identifiers for attribute lookup in the local namespace -- either way subclasses provide no help and only get in the way. One negative effect of the class implementation is that it inherits from unicode and always returns a unicode result even if all of the inputs (mapping values and template) are regular strings. With a function implementation, that can be avoided (returning unicode only if one of the inputs is unicode). The function approach also makes it possible to have keyword arguments (see the example above) as well as a mapping. This isn't a big win, but it is nice to have and reads well in code that is looping over multiple substitutions (mailmerge style): for girl in littleblackbook: print dollarsub(loveletter, name=girl[0].title(), favoritesong=girl[3]) Another minor advantage for a function is that it is easier to lookup in the reference. If a reader sees the % operator being applied and looks it up in the reference, it is going to steer them in the wrong direction. This is doubly true if the Template instantiation is remote from the operator application. Summary for functions: * is more appropriate when there is no state * no unnecessary instantiation * can be applied in a single step * a little easier to learn/use/document * doesn't force result to unicode * allows keyword arguments * easy to find in the docs Raymond ----------- Sample Implementation ------------- def dollarsub(template, mapping=None, **kwds): """A function for supporting $-substitutions.""" typ = type(template) if mapping is None: mapping = kwds def convert(mo): escaped, named, braced, bogus = mo.groups() if escaped is not None: return '$' if bogus is not None: raise ValueError('Invalid placeholder at index %d' % mo.start('bogus')) val = mapping[named or braced] return typ(val) return _pattern.sub(convert, template) def safedollarsub(template, mapping=None, **kwds): """A function for $-substitutions. This function is 'safe' in the sense that you will never get KeyErrors if there are placeholders missing from the interpolation dictionary. In that case, you will get the original placeholder in the value string. """ typ = type(template) if mapping is None: mapping = kwds def convert(mo): escaped, named, braced, bogus = mo.groups() if escaped is not None: return '$' if bogus is not None: raise ValueError('Invalid placeholder at index %d' % mo.start('bogus')) if named is not None: try: return typ(mapping[named]) except KeyError: return '$' + named try: return typ(mapping[braced]) except KeyError: return '${' + braced + '}' return _pattern.sub(convert, template)

16 years, 4 months

Python 3.0 list of goals

by A.M. Kuchling

The thread about bytes() is about a Python 3.0 feature. Guido's presentations have mentioned various changes he'd like to make in 3.0, but there's no master list of things that would change. I think it would be useful to have such a list, because it would focus our language development effort on ones that are steps to 3.0, and maybe people can invent ways to smooth the transition. I've started a list in the Wiki at http://www.python.org/moin/Python3.0 , but should it be a PEP? (PEP 3000, perhaps?) --amk

16 years, 4 months

Alternative placeholder delimiters for PEP 292

by Andrew Durdin

A Yet Simpler Proposal, modifying that of PEP 292 I propose that the Template module not use $ to set off placeholders; instead, placeholders are delimited by braces {}. The following rules for {}-placeholders apply: 1. {{ and }} are escapes; they are replaced with a single { or } respectively. 2. {identifier} names a substitution placeholder matching a mapping key of "identifier". By default, "identifier" must spell a Python identifier as defined in Identifiers and Keywords[1]. No other characters have special meaning. If the left-brace { is unmatched, appears at the end of the string, or is followed by any non-identifier character, a ValueError will be raised at interpolation time[2]. If a single, unmatched right-brace } occurs in the string, a ValueError will be raised at interpolation time. This avoids ambiguity: did the user want a single right-brace, or did they inadvertently omit the left-brace? This will also cause a probably erroneous "{foo}}" or "{{foo}" to raise a ValueError. Rationale There are several reasons for preferring the paired delimiters {} to a single prefixed $: 1. The placeholder name stands out more clearly from its surroundings, due to the presence of a closing delimiter, and also to the fact that the braces bear less resemblance to any alphabetic characters than the dollar sign: "Hello, {name}, how are you?" vs "Hello, $name, how are you?" 2. Only two characters have special meanings in the string, as opposed to three. Additionally, dollar signs are expected to be more often used in templated strings (e.g. for currency values) than braces: "The {item} costs ${price}." vs "The $item costs $$$price." 3. The placeholder syntax is consistent, and does not change even when valid identifier characters follow the placeholder but are not part of the placeholder: "Look at the {plural}s!" vs "Look at the ${plural}s!" 4. The template substitution could be changed in future to support dotted names without breaking existing code. The example below would break if the $-template were changed to allow dotted names: "Evaluate {obj}.__doc__" vs "Evaluate $obj.__doc__" There are two drawbacks to the proposal when compared with $-templates: 1. An extra character must be typed for each placeholder in the common case: "{name}, welcome to {site}!" vs "$name, welcome to $site!" 2. Templated strings containing braces become more complicated: "dict = {{'{key}': '{value}'}}" vs "dict = {'$key': '$value'}" The first is not a real issue; the extra closing braces needed for the placeholder when compared with the number of other characters in the templated string will usually be insignificant. Furthermore, the {}-placeholders require fewer characters to be typed in the less common case when valid identifier characters follow the placeholder but are not part of it. The need for braces in a templated string is not expected to occur frequently. Because of this, the second drawback is considered of minor importance. Reference Implementation If the general nature of feedback on this proposal is positive, or expressive of interest in an implementation, then a reference implementation will be created forthwith. References and Notes [1] Identifiers and Keywords http://www.python.org/doc/current/ref/identifiers.html [2] The requirement for interpolation-time error raising is the same as in PEP 292. Although not a part of this proposal, I suggest that it would be better if the error occured when the Template instance is created. It is worth noting that the PEP 292 reference implementation (in python/nondist/sandbox/string/string/template.py) does not fully conform to the PEP as regards raising errors for invalid template strings: >>> t = Template("This should cause an error: $*") >>> t % {} u'This should cause an error: $*' >>> t = Template("This also should cause an error: $") >>> t % {} u'This also should cause an error: $'

16 years, 4 months

Alternative Implementation for PEP 292: Simple String Substitutions

by Raymond Hettinger

> > * doesn't force result to unicode [Aahz] > This is the main reason I'm +0, pending further arguments. Employing class logic for a function problem is my main reason. Any function can be wrapped in class logic and use an overloaded operator (string.template for example) -- it's invariably the wrong thing to do for reasons of complexity, separation of instantiation and application, and the issues that always arise when an operator is overloaded (a good technique that should only be applied sparingly). [Aahz] > OTOH, I also > like using %, so you'd have to come up with more points to move me > beyond +0. The reasons center around the remoteness of instantiation from application, the differences from current uses of %, and an issue with the % operator itself. When application is remote from instantiation (for instance, when a template argument is supplied to a function), we get several problems. Given a line like "r = t % m", it is hard to verify that the code is correct. Should there be KeyError trapping? Can m be a tuple? Is t a %template, a safetemplate, or dollartemplate? Is the result a str or Unicode object? Should the decision of safe vs regular substitution be made at the point of instantiation or application? The temptation will be to reuse existing code that uses the % operator which unfortunately is not the same (especially with respect to applying tuples, return types, and auto-stringizing). The % operator is hard to search for in the docs and has precedence issues arising from its primary use for modulo arithemetic. Also, using a function makes it possible to use both % formatting and $ formatting (I do have use a case for this). Further, the % operator mnemonically only makes sense with % identifier tags -- it makes less sense with $ tags. Whatever answer is chosen, it should be forward looking and consider that others will want to add functionality (local namespace lookups for example) and that some poor bastards are going to waste time figuring out how to subclass the current implementation. In time, there will likely be more than two of these -- do you want more classes or more functions? Raymond

16 years, 4 months

RE: [Python-Dev] PEP 309 updated slightly

by Moore, Paul

From: Aahz >On Mon, Aug 30, 2004, Peter Harris wrote: >> >> I have updated PEP309 a little, [...] > >> I hope there will still be time to squeeze it in before 2.4 beta 1, and >> if there is any reason why not (apart from there has to be a cut-off >> point somewhere, which I accept), please someone let me know what work >> is still needed to make it ready to go in. > What's PEP309? PEP 309 is the "Partial Function Application" class. (Used to be referred to as "Currying", but that was agreed not to be an accurate name). Paul __________________________________________________________________________ This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. __________________________________________________________________________

16 years, 4 months

[Python-Dev] FW: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26

by Raymond Hettinger

> List some possible reasons why arriving at consensus about > decorators has been so hard (or impossible) to achieve Thanks. I think that was an important contribution to the discussion. At this point, looking at the meta-discussion is likely to best way to help cut through the current morass. +There is no one clear reason why this should be so, but a +few problems seem to be most problematic. A possible social issue is that decorators can be used in a tremendous variety of ways, each of which is only needed or appreciated by small, disjoint groups of users. For instance, applications to ctypes have unique needs that not understood or shared by others. Some users want to use decorators for metaclass magic that scares the socks off of the rest. > + Almost everyone agrees that decorating/transforming a function at > + the end of its definition is suboptimal. One other point of agreement is that no one like having to write the function name three times: def f(): ... f = deco(f) > +* Syntactic constraints. There is an existing (though clumsy) syntax. Most of the proposals are less flexible but more elegant. This trade-off has created much more consternation than if there were no existing ways to apply decorations. +* Overall unfamiliarity with the concept. For people who have a + passing acquaintance with algebra (or even basic arithmetic) or have + used at least one other programming language, much of Python is + intuitive. Very few people will have had any experience with the + decorator concept before encountering it in Python. There's just no + strong preexisting meme that captures the concept. My take on this one is that there are some existing memes from C# and Java and that in the future they will be more widely known. However, there are not good existing, thought-out concepts of what exactly decorators should do in terms of preserving docstrings, attributes, using parameters, etc. Most have only a single application in mind and the full implications (and possible applications) of a given syntax are shrouded in the mists of the future. Like metaclasses, everyone knew they were powerful when they were put in, but no one knew how they would be used or whether they were necessary. In fact, two versions later, we still don't know those answers. Raymond

16 years, 4 months

Re: [Python-Dev] problem with pymalloc on the BeOS port.

by François Revol

Tim Peters <tim.peters(a)gmail.com>: > [François Revol] > > I'm updating the BeOS port of python to include the latest version > > in > > Zeta, the next incarnation of BeOS. > > > > I'm having some problem when enabling pymalloc: > > [zapped] > > 0 bytes originally requested > > The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected. > > The 4 pad bytes at tail=0x80010fb8 are not all FORBIDDENBYTE > > (0xfb): > > at tail+0: 0xdb *** OUCH > > at tail+1: 0xdb *** OUCH > > at tail+2: 0xdb *** OUCH > > at tail+3: 0xdb *** OUCH > > The block was made by call #3688627195 to debug malloc/realloc. > > No it wasn't. The four bytes following the 4 trailing 0xfb hold the > call number, and they're apparently corrupt too. Eh... > > > Fatal Python error: bad trailing pad byte > > > > indeed, there seem to be 4 deadblock marks between the forbidden > > ones, > > while the len is supposed to be null: > > That's reliable. If there actually was a request for 0 bytes (that > is, assuming this pointer isn't just pointing at a random memory > address), the debug pymalloc allocates 16 bytes for it, filled with > > 00000000 fbfbfbfb fbfbfbfb serial > > where "serial" is a big-endian 4-byte "serial number" uniquely > identifying which call to malloc (or realloc) this was. The address > of the second block of fb's is returned. Yes that's what I deduced from the code of pymalloc. > > > python:dm 0x80010fb8-8 32 > > 80010fb0 00000000 fbfbfbfb dbdbdbdb fbfbdbdb > > ................. > > > > 80010fc0 0100fbfb 507686ef 04000000 fbfbfbfb > > .......vP........ > > 80010fd0 8013cbc8 fbfbfbfb 44ee0100 ffed0100 > > ............D.... > > > > Any clue ? > > Try gcc without -O. Nobody has reported anything like this before -- > you're in for a lot of fun <wink>. > OK, tried -O0 -g but same result. I suspect it might be a bad interaction with fork(), as it crashes in a child, quite badly, as no images are repported as loaded in the team (= no binary are mapped in the process), however the areas are there (= pages). Now, I don't see why malloc itself would give such a result, it's pyMalloc which places those marks, so the thing malloc does wouldn't place them 4 bytes of each other for no reason, or repport 0 bytes where 4 are allocated. François.

16 years, 4 months

Jump to page:

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev August 2004