Perhaps this is more approprate for python-list but I looks like a
bug to me. Example code:
class A:
def __str__(self):
return u'\u1234'
'%s' % u'\u1234' # this works
'%s' % A() # this doesn't work
It will work if 'A' subclasses from 'unicode' but should not be
necessary, IMHO. Any reason why this shouldn't be fixed?
Neil
Hi.
[Mark Hammond]
> The point isn't about my suffering as such. The point is more that
> python-dev owns a tiny amount of the code out there, and I don't believe we
> should put Python's users through this.
>
> Sure - I would be happy to "upgrade" all the win32all code, no problem. I
> am also happy to live in the bleeding edge and take some pain that will
> cause.
>
> The issue is simply the user base, and giving Python a reputation of not
> being able to painlessly upgrade even dot revisions.
I agree with all this.
[As I imagined explicit syntax did not catch up and would require
lot of discussions.]
[GvR]
> > Another way is to use special rules
> > (similar to those for class defs), e.g. having
> >
> > <frag>
> > y=3
> > def f():
> > exec "y=2"
> > def g():
> > return y
> > return g()
> >
> > print f()
> > </frag>
> >
> > # print 3.
> >
> > Is that confusing for users? maybe they will more naturally expect 2
> > as outcome (given nested scopes).
>
> This seems the best compromise to me. It will lead to the least
> broken code, because this is the behavior that we had before nested
> scopes! It is also quite easy to implement given the current
> implementation, I believe.
>
> Maybe we could introduce a warning rather than an error for this
> situation though, because even if this behavior is clearly documented,
> it will still be confusing to some, so it is better if we outlaw it in
> some future version.
>
Yes this can be easy to implement but more confusing situations can arise:
<frag>
y=3
def f():
y=9
exec "y=2"
def g():
return y
return y,g()
print f()
</frag>
What should this print? the situation leads not to a canonical solution
as class def scopes.
or
<frag>
def f():
from foo import *
def g():
return y
return g()
print f()
</frag>
[Mark Hammond]
> > This probably won't be a very popular suggestion, but how about pulling
> > nested scopes (I assume they are at the root of the problem)
> > until this can be solved cleanly?
>
> Agreed. While I think nested scopes are kinda cool, I have lived without
> them, and really without missing them, for years. At the moment the cure
> appears worse then the symptoms in at least a few cases. If nothing else,
> it compromises the elegant simplicity of Python that drew me here in the
> first place!
>
> Assuming that people really _do_ want this feature, IMO the bar should be
> raised so there are _zero_ backward compatibility issues.
I don't say anything about pulling nested scopes (I don't think my opinion
can change things in this respect)
but I should insist that without explicit syntax IMO raising the bar
has a too high impl cost (both performance and complexity) or creates
confusion.
[Andrew Kuchling]
> >Assuming that people really _do_ want this feature, IMO the bar should be
> >raised so there are _zero_ backward compatibility issues.
>
> Even at the cost of additional implementation complexity? At the cost
> of having to learn "scopes are nested, unless you do these two things
> in which case they're not"?
>
> Let's not waffle. If nested scopes are worth doing, they're worth
> breaking code. Either leave exec and from..import illegal, or back
> out nested scopes, or think of some better solution, but let's not
> introduce complicated backward compatibility hacks.
IMO breaking code would be ok if we issue warnings today and implement
nested scopes issuing errors tomorrow. But this is simply a statement
about principles and raised impression.
IMO import * in an inner scope should end up being an error,
not sure about 'exec's.
We will need a final BDFL statement.
regards, Samuele Pedroni.
I thought it would be nice to try to improve the mimetypes module by having
it, on Windows, query the Registry to get the mapping of filename extensions
to media types, since the mimetypes code currently just blindly checks
posix-specific paths for httpd-style mapping files. However, it seems that the
way to get mappings from the Windows registry is excessively slow in Python.
I'm told that the reason has to do with the limited subset of APIs that are
exposed in the _winreg module. I think it is that EnumKey(key, index) is
querying for the entire list of subkeys for the given key every time you call
it. Or something. Whatever the situation is, the code I tried below is way
slower than I think it ought to be.
Does anyone have any suggestions (besides "write it in C")? Could _winreg
possibly be improved to provide an iterator or better interface to get the
subkeys? (or certain ones? There are a lot of keys under HKEY_CLASSES_ROOT,
and I only need the ones that start with a period). Should I file this as a
feature request?
Thanks
-Mike
from _winreg import HKEY_CLASSES_ROOT, OpenKey, EnumKey, QueryValueEx
i = 0
typemap = {}
try:
while 1:
subkeyname = EnumKey(HKEY_CLASSES_ROOT, i)
try:
subkey = OpenKey(HKEY_CLASSES_ROOT, subkeyname)
if subkeyname[:1] == '.':
data = QueryValueEx(subkey, 'Content Type')[0]
print subkeyname, '=', data
typemap[subkeyname] = data # data will be unicode
except EnvironmentError, WindowsError:
pass
i += 1
except WindowsError:
pass
Would anyone care to check this cookielib.py patch in before 2.4b1?
http://python.org/sf/1028908
Quote from above:
The patch contains uncontroversial changes to cookielib and associated
modules, documentation and tests. Would be unfortunate not to have these
in 2.4.0.
[...SNIP...]
John
Current test_shutil is apparently trying to do something that can't
work on Windows, so it fails:
test_shutil
test test_shutil failed -- Traceback (most recent call last):
File "C:\Code\python\lib\test\test_shutil.py", line 28, in test_on_error
shutil.rmtree(TESTFN, onerror=self.check_args_to_onerror)
File "C:\Code\python\lib\shutil.py", line 168, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "C:\Code\python\lib\test\test_shutil.py", line 36, in
check_args_to_onerror
self.assertEqual(func, os.remove)
AssertionError: <built-in function rmdir> != <built-in function remove>
1 test failed:
test_shutil
After that, it leaves behind temp directories that cause other tests
to fail. Sorry, I've got other problems on my stack, and can't make
time for this one.
Lib\email\test\data\PyBanner048.gif wasn't marked binary in CVS. It
is now. If you're on Windows, delete your copy, then cvs up to
refetch an unmangled copy.
[Followups to python-dev, please.]
[Michael Hohn]
> > > under python 2.2, the pickle/unpickle sequence incorrectly restores
> > > a larger data structure I have.
> > >
> > > Under Python 2.3, these structures now give an explicit exception from
> > > Pickle.memoize():
> > > assert id(obj) not in self.memo
> > >
[Tim Peters]
> > Assertions are never expected to fail, so "something impossible
> > happened" when they do fail.
[Michael Hohn]
> Here is a code sample that shows the problem I ran into:
Summary for the OP: This is a bug in Python. Using cPickle won't help,
but if you don't subclass builtin container types others than list,
dict, and tuple, using pickle protocol 2 should work. The rest of this
message is for python-dev.
The simplest breaking case is:
t = type('t', (list,), {})
obj = t()
obj.append(obj)
pickle.dumps(obj)
[infinite recursion]
The subclass causes save_reduce to be used instead of save_list. For
proto < 2, copy_reg._reduce_ex returns (_reconstructor, list(a)), and
the args--list(a)--cycle back through obj. Initially it looks like this
should be okay, but the args are saved before obj is memoized, and obj
can't be memoized until REDUCE can be executed with the args--and
there's the cycle. It is even more obviously impossible from the
unpickler's perspective because it has to call _reconstructor([obj]) to
create obj!
There are two separate problems:
1. Any __reduce__ implementation that returns args that cycle back
through the object it tried to reduce hasn't done its job. As
described above, _reduce_ex is one such implementation. reduce_2
avoids this by using the listitems and dictitems parameters. Since
that's a pickler-side feature it can be used in _reduce_ex too. The
basetype(obj) hook (documented in PEP 307) would remain for
immutable bases; it doesn't work for containers, but user-defined
containers already have to implement their own reduce functions.
POC patch: http://www.trit.org/~dima/home/reduce_ex.diff
At least the set and deque types also have this problem.
2. The pickle implementations don't detect reduction cycles. Pickling
an instance of this obviously broken class causes an infinite
recursion:
class evil(object):
def __reduce__(self):
return evil, (self,)
It's easy to detect this case. POC patch for the pickle module:
http://www.trit.org/~dima/home/redcycle.diff
BTW, the failed assert the OP is seeing happens when the cycle goes
through another object:
t = type('t', (list,), {})
obj = t()
d = {'obj': obj}
obj.append(d)
pickle.dumps(obj)
[AssertionError]
cPickle has the same problem, but it lacks the assert, so it writes
garbage instead:
new = cPickle.loads(cPickle.dumps(obj))
new[0]['obj'] is new -> False # wrong
obj[0]['obj'] is obj -> True # right
This makes the reduction cycle check (#2 above) more than just cosmetic
since if cPickle had that assert (it should) it would've been a crash.
Right now it's garbage output instead, which is arguably worse.
Formally complete versions of the above patches will be on SF tomorrow
unless someone suggests better alternatives.
Dima.
On Thu, Oct 21, 2004 at 12:49:19PM +0400, Oleg Broytmann wrote:
>
> BTW, just installing is not enough, even when it is come with Python
> distribution. Installing a newer version of BerkeleyDB breaks older
> databases due to incompatible file formats.
fwiw its possible to fix this in the bsddb code. someone willing to do
the exception catching database backing-up + auto db_upgrade() work should
submit a patch. (i added backing-up as its rude to auto-convert someones
database for them to a version their old software can no longer read)
-g
