Mailman 3 June 2001 - Python-Dev

Marshal bug in 2.1?
by Christian Tismer June 2, 2001

June 2, 2001

Hi friends, there is a script which generates encrypted passwords for Starship users. There is a series of marshal, zlib and base64 calls, which is reversed by the script. Is there a known bug in Marshal, or should I start the debugger now? The passwphrase for the attached script is "hey". cheers - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net/ 14163 Berlin : PGP key -> http://wwwkeys.pgp.net/ PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF where do you want to jump today? http://www.stackless.com/ import marshal,base64,zlib exec marshal.loads(zlib.decompress(base64.decodestring(""" eJytVM+PGzUUfs6PzWZYwapAqbbAuiyF6Yqsqt2iomq1HGkvuQQJaS+pM3YzbjP2yHY6CdrVHNr+ Exz5L/gn4MidC2f+Az5Pkq0QlFMnmTf2s+d73/vmPWeEq43b/wxT498mSXSOwbskGZ0zqm+QbNF5 i+o9km16idU21bdIdUh26GmLrCRWf0ayS8+6dN6l+oAU0XcP689JbZHcohfA6VF9mxQj1SbVi57r 2PAFqS7p7bVH9+kFkew1mDvA/JJUCziGEYs3AozS7ch1yIiSg7dwJfjxzCkRVFml4Q7ng8F6zgUv hfeVdZLzJ84WXJgln+rnyvCgFuEIbzoV5s54/g3PcuFEFpTzvMp1lnPhFM9sUc6DklwboEmF5UIb 7YPO8PJkHvhz5ZbcWDOYaaOE45VYrmI18N/n2sctXlvDMczmPthC/wjEJ9bxUrtFTOBt6OAPoqSH h4c85MqrdUaeT1SoFDIenJ0OmpyWdu5AxDllwmuB8GLC33gNzm7700EytBWfA3s0esiD5TM7hTAY +IBIuS6PymXIrTkyKiRYjKL5+MI607nXZsrVAjLPlpHmFck0m+lyYgWIOAXRC2UkNHowuJMII+Mm M10zv2K8QosojUvy0tmpE0WyomQLFfK4o7BIGgUhxWSmjhJ/F/U3CdVX/BHPRKyE2SwiA0mEVQgI g49agXtmIVMWbmWMOvi1yZexyfaovhmb7BnRJWsGjC7RXh/TBZqgFdsO3XCJJvuELtqkO3RB0cPq T5v5VmyTSwDt00WLdI/CduxQNGbc14pNGm2H+Ajgo7SLoEPfhz25e3x8cv/eyX0wYuADRjepAQpE ga3jIP514H2E4SiNZ8NQj2E1h2nmPposd80TYnrUDi3SaFdD/37c8O9q9bF7T2eimEhxtk8+Hj6N 0XEh7W+wC/m134qT4PANGpdRVYMtm4V5KdGijSM0DqmnygffwfCp1WaFIsq0s+EU/gt4Bfh/ZDdn wx75JJ6U7EN2je2y91izOh4XQpvxeOj3MStnSqC88f1RsqtSiMXKy9zB/8DvYs/jH/46fWR+q3+v fv3lz5/+eJUmm5ylzRr6eB5vBif/4LAOaUShxuOrdKJoTlRjbXDWNN6wCFeSvdYmbcR+U65RiW9R Dh/gufNOP+m3dnq7bIdtI9VrbJ/9DYOcdyU= """)))

3 5

Re: [Python-Dev] another dict crasher
by Michael Hudson June 2, 2001

June 2, 2001

"Tim Peters" <tim.one(a)home.com> writes: > [Michael Hudson] > > Actually this crash was dict_print (I always forget about tp_print...). > > We all should <wink>. > > > It's pretty easy to mend: [snip] > > I'm not sure this stops still more Machiavellian behaviour from > > crashing the interpreter, > > Alas, it doesn't. No, that's what my "dict[Machiavelli()] = Machiavelli()" example was demonstrating. If noone beats me to it, I'll post a better fix to sf next week, complete with test-cases and suitably "encouraging" comments. I can't easily see other examples of the problem; there certainly might be things you could do with comparisons that could trigger crashes, but that code's so hairy that it's almost impossible for me to be sure. There are two ways of constructing a software design: one way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies. -- C. A. R. Hoare > > and you can certainly get items being printed more than once or not > > at all. I'm not sure this last is a problem; > > Those don't matter: in a long tradition, we buy "safety" not only at the > cost of bloating the code, but also by making the true behavior in case of > mutation unpredictable & inexplicable. This is what I thought. [snip] > > if the user's being this contrary there's only so much we can > > do to help him or her. > > I'd prefer a similar internal immutable-dict trick that raised an exception > if the user was pushing Python into a corner where "blow up or do something > baffling" were its only choices. That would render the original example > illegal, of course. But would that be a bad thing? It's hard to see how. > What *should* it mean when the user invokes an operation on a > container and mutates the container during that operation? I don't think there's a meaning you can attach to this kind of behaviour. The "immutable dict trick" looks better the more I think about it, but I guess that will have to wait until Guido gets back from the sun... Cheers, M. -- incidentally, asking why things are "left out of the language" is a good sign that the asker is fairly clueless. -- Erik Naggum, comp.lang.lisp

2 1

Re: [Python-Dev] another dict crasher
by Michael Hudson June 2, 2001

June 2, 2001

Michael Hudson <mwh(a)python.net> writes: > Adapted from a report on comp.lang.python from Wolfgang Lipp: [snip] > segfaults both 2.1 and current (well, maybe a day old) CVS. Haven't > tried Tim's latest patch, but I don't believe that will make any > difference. > > It's obvious <wink> what's happening; the dict's resizing inside the > for loop in dict_repr and the ep pointer is dangling. Actually this crash was dict_print (I always forget about tp_print...). It's pretty easy to mend: *** dictobject.c Fri Jun 1 13:08:13 2001 --- dictobject.c-fixed Fri Jun 1 12:59:07 2001 *************** *** 793,795 **** any = 0; ! for (i = 0, ep = mp->ma_table; i < mp->ma_size; i++, ep++) { if (ep->me_value != NULL) { --- 793,796 ---- any = 0; ! for (i = 0; i < mp->ma_size; i++) { ! ep = &mp->ma_table[i]; if (ep->me_value != NULL) { *************** *** 833,835 **** any = 0; ! for (i = 0, ep = mp->ma_table; i < mp->ma_size && v; i++, ep++) { if (ep->me_value != NULL) { --- 834,837 ---- any = 0; ! for (i = 0; i < mp->ma_size && v; i++) { ! ep = &mp->ma_table[i]; if (ep->me_value != NULL) { I'm not sure this stops still more Machiavellian behaviour from crashing the interpreter, and you can certainly get items being printed more than once or not at all. I'm not sure this last is a problem; if the user's being this contrary there's only so much we can do to help him or her. Cheers, M. -- I also feel it essential to note, [...], that Description Logics, non-Monotonic Logics, Default Logics and Circumscription Logics can all collectively go suck a cow. Thank you. -- http://advogato.org/person/Johnath/diary.html?start=4

2 1

Re: [Python-Dev] another dict crasher
by Michael Hudson June 2, 2001

June 2, 2001

"Tim Peters" <tim.one(a)home.com> writes: > The dict code has even more holes and in more places, but they're > generally much harder to provoke, so they've gone unnoticed for 10 > years. All in all, seemed like a good tradeoff to me <wink>. Are you suggesting that we should just leave these crashers in? They're not *particularly* hard to provoke if you know the implementation - and I was inspired to look for them by someone's report of actually running into one. Cheers, M. -- Java sucks. [...] Java on TV set top boxes will suck so hard it might well inhale people from off their sofa until their heads get wedged in the card slots. --- Jon Rabone, ucam.chat

2 1

another dict crasher
by Michael Hudson June 1, 2001

June 1, 2001

Adapted from a report on comp.lang.python from Wolfgang Lipp: class Child: def __init__(self, parent): self.__dict__['parent'] = parent def __getattr__(self, attr): self.parent.a = 1 self.parent.b = 1 self.parent.c = 1 self.parent.d = 1 self.parent.e = 1 self.parent.f = 1 self.parent.g = 1 self.parent.h = 1 self.parent.i = 1 return getattr(self.parent, attr) class Parent: def __init__(self): self.a = Child(self) print Parent().__dict__ segfaults both 2.1 and current (well, maybe a day old) CVS. Haven't tried Tim's latest patch, but I don't believe that will make any difference. It's obvious <wink> what's happening; the dict's resizing inside the for loop in dict_repr and the ep pointer is dangling. By the time we've shaken all of these out of dictobject.c it's going to be pretty close to free-threading safe, I'd have thought. reentrancy-sucks-ly y'rs M. -- But since I'm not trying to impress anybody in The Software Big Top, I'd rather walk the wire using a big pole, a safety harness, a net, and with the wire not more than 3 feet off the ground. -- Grant Griffin, comp.lang.python

4 5

METH_NOARGS calling convention
by Martin von Loewis June 1, 2001

June 1, 2001

The patch http://sourceforge.net/tracker/?func=detail&atid=305470&aid=427190&group_id… introduces two new calling conventions, METH_O and METH_NOARGS. The rationale for METH_O has been discussed already; the rationale for METH_NOARGS is that it allows a convient simplification (plus a marginal speed-up) of functions which do either PyArg_NoArgs(args) or PyArg_ParseTuple(args, ":function_name"). Now, one open issue is whether the METH_NOARGS functions should have a signature of PyObject * (*unaryfunc)(PyObject *); or of PyObject *(*PyCFunction)(PyObject *, PyObject *); which then would be called with a NULL second argument; the first argument would be self in either case. IMO, the advantage of passing the NULL argument is that NOARGS methods don't need to be cast into PyCFunction in the method table; the advantage of the second approach is that it is clearer in the function implementation. Any opinions which signature to use? Regards, Martin

4 3

Re: [Python-Dev] another dict crasher
by Michael Hudson June 1, 2001

June 1, 2001

Warning! VERY SICK CODE INDEED ahead! barry(a)digicool.com (Barry A. Warsaw) writes: > >>>>> "MH" == Michael Hudson <mwh(a)python.net> writes: > > MH> segfaults both 2.1 and current (well, maybe a day old) CVS. > MH> Haven't tried Tim's latest patch, but I don't believe that > MH> will make any difference. > > That is highly, highly nasty. Not as nasty as this, though: dict = {} # let's force dict to malloc its table for i in range(1,10): dict[i] = i class Machiavelli: def __repr__(self): dict.clear() print # doesn't crash without this. don't know why return `"machiavelli"` def __hash__(self): return 0 dict[Machiavelli()] = Machiavelli() print dict gives, even with my posted patch to dictobject.c $ ./python crash2.py { Segmentation fault (core dumped) Any ideas what the above code should do? (Other than use the secret PSU website to hire a hitman and shoot whoever wrote the code, I mean). Cheers, M. -- Well, yes. I don't think I'd put something like "penchant for anal play" and "able to wield a buttplug" in a CV unless it was relevant to the gig being applied for... -- Matt McLeod, alt.sysadmin.recovery

2 1

Damn... I think I might have just muffed a checkin
by Skip Montanaro June 1, 2001

June 1, 2001

I just updated httplib.py to expand the list of names in its __all__ list. I was operating on version 1.34. After the checkin I am looking at version 1.34.2.1. I see that Lib/CVS/Tag exists in my directory tree and says "release21-maint". Did I muff it? If so, how should I do an unmuff operation? Skip

4 6

Re: [Jython-dev] Using PyChecker in Jython
by Samuele Pedroni June 1, 2001

June 1, 2001

Hi. [Neal Norwitz] > Hello! > > I have created a program PyChecker to perform Python source code checking. > (http://pychecker.sourceforge.net). > > PyChecker is implemented in C Python and does some "tricky" things. > It doesn't currently work in Jython due to the module dis (disassemble code) > not being available in Jython. > > Is there any fundamental problem with getting PyChecker to work under Jython? > > Here's a high-level overview of what PyChecker does: > > imp.find_module() > imp.load_module() > for each object in dir(module): > # object can be a class, function, imported module, etc. > for each instruction in disassembled byte code: > # handle each instruction appropriately > > This hides a lot of details, but I do lots of things like getting the code objects from the classes, methods, and > functions, look at the arguments > in functions, etc. > > Is it possible to make work in Jython? Easy? > > Thanks for any guidance, > Neal It would be great - really - but about easy? As easy as making PyChecker working on source code without using dis and without importing/executing modules and their top defs, I think there will be no dis support on jython side (we produce java bytecode and getting "back" to python vm bytecode would be very tricky, not very elegant, etc. ) any time soon <wink>. Seriously, two possible workaround hacks (they are also not very easy), this is just after small brainstorming and ignoring the concrete needs and code of PyChecker: +) more elegant one, but maybe still too difficult or requiring too much work: let PyChecker run under CPython even when checking jython code, jython code can compile down to py vm bytecode but then does not run: why? java classes imports and the jython specific builtin modules (not so many) So one needs to implement a sufficient amount of python (an import hook, etc) code that does the minimal partial evalution required and the required amount of loading&introspection on java, jython specific stuff in order to have the imports work and PyChecher feeded with the things it needs. This means dealing with the java class format, or a two passes approach: run the code under jython in order to gather the information needed to load it succesfully under python. If the top level code contains conditionals that depend on jython stuff this could be hard, but one can ignore that (at least for starting). Clearly the main PyChecker loop would require some adaptation, and maybe include some logic to check some jython specific stuff (subclassing from java, etc). *) let an adapted PyChecker run under jython, obtain someway the needed py vm bytecode stream from a source -> py vm bytecode compiler written in python (such a thing exists - if I remember well) . And similar ideas ... regards, Samuele Pedroni.

1 0

re: %b format
by Greg Wilson June 1, 2001

June 1, 2001

My thanks to everyone who commented on the idea of adding a binary format specifier to Python. I'll volunteer to draft the PEP --- volunteers for a co-author? Greg ----------------------------------------------------------------------------------------------------------------- The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Baltimore Technologies plc will not be liable for direct, special, indirect or consequential damages arising from alteration of the contents of this message by a third party or as a result of any virus being passed on. In addition, certain Marketing collateral may be added from time to time to promote Baltimore Technologies products, services, Global e-Security or appearance at trade shows and conferences. This footnote confirms that this email message has been swept by Baltimore MIMEsweeper for Content Security threats, including computer viruses.

1 0