Mailman 3 April 2005 - Python-Dev

Re: scope-collapse (was: anonymous blocks)
by Robert Brewer April 27, 2005

April 27, 2005

[Jim Jewett] > (2) Add a way to say "Make this function I'm calling > use *my* locals and globals." This seems to meet all > the agreed-upon-as-good use cases, but there is disagreement > over how to sensibly write it. The calling function is > the place that could get surprised, but people who want > thunks seem to want the specialness in the > called function. [Guido] > I think there are several problems with this. First, it looks > difficult to provide semantics that cover all the corners for the > blending of two namespaces. What happens to names that have a > different meaning in each scope? [Jim] > Programming error. Same name ==> same object. [Guido] > Sounds like a recipe for bugs to me. At the very least it is a total > breach of abstraction, which is the fundamental basis of the > relationship between caller and callee in normal circumstances. The > more I understand your proposal the less I like it. [Jim] > If a function is using one of _your_ names for something > incompatible, then don't call that function with collapsed > scope. The same "problem" happens with globals today. > Code in module X can break if module Y replaces (not shadows, > replaces) a builtin with an incompatible object. > > Except ... > (E.g. 'self' when calling a method of > another object; or any other name clash.) > > The first argument of a method *might* be a special case. It seems > wrong to unbind a bound method. On the other hand, resource > managers may well want to use unbound methods for the called > code. Urg. Please, no. If you're going to blend scopes, the callee should have nothing passed to it. Why would you possibly want it when you already have access to both scopes which are to be blended? [Guido] > Are the globals also blended? How? [Jim] > Yes. The callee does not even get to see its normal namespace. > Therefore, the callee does not get to use its normal name > resolution. [Guido] > Another breach of abstraction: if a callee wants to use an imported > module, the import should be present in the caller, not in the callee. Yes, although if a callee wants to use a module that has not been imported by the caller, it should be able to do so with a new import statement (which then affects the namespace of the caller). [Guido again] > It really strikes me as an endless source of errors that these > blended-scope callees (in your proposal) are ordinary > functions/methods, which means that they can *also* be called without > blending scopes. Having special syntax to define a callee intended for > scope-blending seems much more appropriate (even if there's also > special syntax at the call site). Agreed. They shouldn't be ordinary functions at all, in my mind. That means one can also mark the actual call on the callee side, instead of the caller side; in other words, you wouldn't need a "collapse" keyword at all if you formed the callee with a "defmacro" or other (better ;) keyword. I guess if y'all find it surprising, you could keep "collapse". [Jim] > If the name normally resolves in locals (often inlined to a > tuple, today), > it looks in the shared scope, which is "owned" by the > caller. This is > different from a free variable only because the callee can > write to this > dictionary. [Guido] > Aha! This suggests that a blend-callee needs to use different bytecode > to avoid doing lookups in the tuple of optimized locals, since the > indices assigned to locals in the callee and the caller won't match up > except by miracle. [Guido] > Third, I expect that if we solve the first two > problems, we'll still find that for an efficient implementation we > need to modify the bytecode of the called function. [Jim] > Absolutely. Even giving up the XXX_FAST optimizations would > still require new bytecode to not assume them. (Deoptimizing > *all* functions, in *all* contexts, is not a sensible tradeoff.) I'm afraid I'm only familiar with CPython, but wouldn't callee locals just map to XXX_FAST indices via the caller's co_names tuple? Remapping jump targets, on the other hand, would be something to quickly ban. You shouldn't be able to write trash like: defmacro keepgoing: else: continue [Guido] > Try to make sure that it can be used in a "statement context" > as well as in an "expression context". ... > I'm trying to sensitize you to potential uses like this: > > def foo(b): > c=a > def bar(): > a = "a1" > print collapse foo("b1") If the callees aren't real functions and don't get passed anything, the "sensible" approach would be to disallow expression-context use of them. Rewrite the above to: defcallee foo: c = a def bar(): a = "a1" collapse foo print c Robert Brewer MIS Amor Ministries fumanchu(a)amor.org

2 1

Re: RE: [Python-Dev] defmacro (was: Anonymous blocks)
by flaig＠sanctacaris.net April 27, 2005

April 27, 2005

Actually I was thinking of something related the other day: Wouldn't it be nice to be able to define/overload not only operators but also control structures? That way Python's core language could be kept simple and free of "featuritis" while at the same time everyone who desires a match/case or repeat/until statement, or anything more sophisticated, could implement it for himself. If you suspect that good old Lisp is on my mind, you are probably right :) . Actually, the idea of a programming language whose structures can be adapted to everyone's personal style is still very appealing to me. I have no very distinct ideas about how such a thing might be designed (and still less whether it could be made to work efficiently), but perhaps somewhat like this (just to clarify along which paths my thoughts are currently moving): structure whaddayacallit: # name only as a comment def opening_clause: statements def alternative_clause_1: statements def *alternative_clause_2: # the asterisk to indicate that this may occur several times statements def closing-clause: statements e.g.: structure multiple_switch: condition = None def switch(self, c): # condition must be passed as a lambdoid function self.condition, self.finished = c, False def *case(self, x, statements): # so must the statements subordinate to the new "case" expression if self.condition( x ): statements() self.finished = True break structure def otherwise(self, statements); if not self.finished: statements() and the application: switch my_favourite_language: case Python: print "Hi Guido" case Perl: print "Hi Larry" otherwise: print "Hi pleb" At least to me, this has a definitively macroish flavour... and not in the #dumbdown style of C. Or rather say, macros might be a generalized way to achieve this, if they are intelligently designed. (<= That at least shouldn't be the problem, since this is the Python community and not M$'s development department :-) .) Do you think any of this might make sense? -- Rüdiger Marcus PS. Aahz: When describing Ruby as the "antithesis" of Python recently I was thinking in Laskerian rather than Hegelian terms... the differences are not really big, but Ruby has always been positioned as a deliberate challenge to Python. Date: Mon, 25 Apr 2005 09:42:54 -0700 > From: Michael Chermside <mcherm(a)mcherm.com> > Subject: RE: [Python-Dev] defmacro (was: Anonymous blocks) > To: python-dev(a)python.org > Cc: jimjjewett(a)gmail.com > Message-ID: <20050425094254.wf6wbmkg0pwkocwo(a)mcherm.com> > Content-Type: text/plain; charset=ISO-8859-1 > > Jim Jewett writes: > > As best I can tell, the anonymous blocks are used to take > > care of boilerplate code without changing the scope -- exactly > > what macros are used for. > > Folks, I think that Jim is onto something here. > > I've been following this conversation, and it sounds to me as if we > are stumbling about in the dark, trying to feel our way toward something > very useful and powerful. I think Jim is right, what we're feeling our > way toward is macros. > > The problem, of course, is that Guido (and others!) are on record as > being opposed to adding macros to Python. (Even "good" macros... think > lisp, not cpp.) I am not quite sure that I am convinced by the argument, > but let me see if I can present it: > > Allowing macros in Python would enable individual programmers or > groups to easily invent their own "syntax". Eventually, there would > develop a large number of different Python "dialects" (as some > claim has happened in the Lisp community) each dependent on macros > the others lack. The most important casualty would be Python's > great *readability*. > > (If this is a strawman argument, i.e. if you know of a better reason > for keeping macros OUT of Python please speak up. Like I said, I've > never been completely convinced of it myself.) > > I think it would be useful if we approached it like this: either what > we want is the full power of macros (in which case the syntax we choose > should be guided by that choice), or we want LESS than the full power > of macros. If we want less, then HOW less? > > In other words, rather than hearing what we'd like to be able to DO > with blocks, I'd like to hear what we want to PROHIBIT DOING with > blocks. I think this might be a fruitful way of thinking about the > problem which might make it easier to evaluate syntax suggestions. And > if the answer is that we want to prohibit nothing, then the right > solution is macros. > > -- Michael Chermside > === Chevalier Dr. Dr. Ruediger Marcus Flaig Institute for Immunology University of Heidelberg Im Neuenheimer Feld 305, D-69120 Heidelberg, FRG <flaig(a)sanctacaris.net> -- Diese E-Mail wurde mit http://www.mail-inspector.de verschickt Mail Inspector ist ein kostenloser Service von http://www.is-fun.net Der Absender dieser E-Mail hatte die IP: 129.206.124.135

2 1

scope-collapse
by Jim Jewett April 27, 2005

April 27, 2005

[Jim Jewett] >> >> (2) Add a way to say "Make this function I'm calling use *my* locals >> >> and globals." This seems to meet all the agreed-upon-as-good use >> >> cases, but there is disagreement over how to sensibly write it. [Guido] >> > What happens to names that have a >> > different meaning in each scope? [Jim] >> Programming error. Same name ==> same object. > Sounds like a recipe for bugs to me. At the very least it is a total > breach of abstraction, which is the fundamental basis of the > relationship between caller and callee in normal circumstances. Yes. Collapsing scope is not a good idea in general. But there is no way to avoid (some version of) it under the thunk or even the resource-manager suggestions. I interpret that to mean that either The code *can* get ugly and you rely on conventions. or These constructs should not be added to the language. The pretend-it-is-a-generator proposals try to specify that only certain names will be shared, in only certain ways. That might work (practicality beats purity) but I suspect it will evolve into a wart. It won't be quite strong enough to solve the problem completely (particularly with multiple blocks), but it will be strong enough to obfuscate when mishandled. >> Yes. The callee does not even get to see its normal namespace. >> Therefore, the callee does not get to use its normal name resolution. > This seems to me to repeat all the mistakes of the dynamic scoping > of early Lisps (including GNU Emacs Lisp I believe). With one exception -- the caller must state explicitly that the collapse it happening, and even then, it only goes down one level at a time. Still an ugly tool, but at least not an ugly surprise. > It really strikes me as an endless source of errors that these > blended-scope callees (in your proposal) are ordinary > functions/methods, which means that they can *also* be called without > blending scopes. Having special syntax to define a callee intended for > scope-blending seems much more appropriate (even if there's also > special syntax at the call site). This might well be a good restriction. The number of times it causes annoyance (why do I have to code this twice?) should be outweighed by the number of times it saves a surprise (oops -- those functions both defined the same keyword argument). >> If the name normally resolves in locals (often inlined to a tuple, today), >> it looks in the shared scope, which is "owned" by the caller. This is >> different from a free variable only because the callee can write to this >> dictionary. > Aha! This suggests that a blend-callee needs to use different bytecode > to avoid doing lookups in the tuple of optimized locals Yes. I believe the translation is mechanical, so that the compiler could choose (or create) the right version based on the caller, but ... I agree that making them a separate kind of callable would simplify things. > (I meant this to read "does not make sense for all callables".) > (And I presume you read it that way. :-) nah... I think it takes special justification to do use anything but duck typing in python. If functions can intersperse with boilerplate, than other callables (and even other suites, such as class definitions) should be able to do the same. But I also agree that it makes sense to wait until it can be done sensibly. Just as @decorator only applies to functions (even if the specific decorator could accept something else), this interspersing should probably not apply to non-functions until the "but I can't use a function" use cases are clear. >> For a given simple algorithm, interpeted python is generally slower >> than compiled C, but we write in python anyhow -- it is fast enough, >> and has other advantages. The same is true of anything that lets >> me not cut-and-paste. > Whatever. Any new feature that causes a measurable slowdown for code > that does *not* need the feature has a REALLY hard time getting > accepted, Agreed. But the scope-collapse penalty is restricted to the caller (which ordered the collapse) and the immediate callees during the collapsed call. >> I'm not sure I understand this. The preferred way would be >> to just stick the keyword before the call. Using 'collapse', it >> would look like: # (Added comment to make the ugliess potential more explicit) >> def foo(b): # Yes, parameters are in the namespace. >> c=a >> def bar(): >> a="a1" >> collapse foo("b1") >> print b, c # prints "b1", "a1" >> a="a2" >> foo("b2") # Not collapsed this time >> print b, c # still prints "b1", "a1" > I'm trying to sensitize you to potential uses like this: > def bar(): > a = "a1" > print collapse foo("b1") In this case, it would print None, as foo didn't bother to return anything. > but I suppose you could treat it like the 'global' keyword >> def bar(): >> a="a1" >> collapse foo # forces foo to always collapse when called within bar >> foo("b1") >> print b, c # prints "b1", "a1" >> a="a2" >> foo("b2") # still collapsed >> print b, c # now prints "b2", "a2" > Would make more sense if the collapse keyword was at the module level. ??? Are you suggesting that everything defined in the module must live in a single namespace, just because the collapse was wanted in one place? -jJ

2 1

defmacro (was: Anonymous blocks)
by Jim Jewett April 27, 2005

April 27, 2005

>> >> (3) Add macros. We still have to figure out how to limit their obfuscation. >> > nobody has given even a *remotely* >> > plausible mechanism for how exactly you would get code executed at >> > compile time. >> macros can (and *possibly* should) be evaluated at run-time. > We must still have very different views on what a macro is. In a compiled language, it is (necessarily) compiled. In an interpreted language, it doesn't have to be. > After a macros is run, there is new syntax that needs to be parsed > and compiled to bytecode. ... anything that requires invoking the > compiler each time a macro is used will be so slow that nobody will > want to use it. I had been thinking that the typical use would be during function (or class) definition. The overhead would be similar to that of decorators, and confined mostly to module loading. I do see your point that putting a macro call inside a function could be slow -- but I'm not sure that is a reason to forbid it. >> Even if the macros just rerun the same boilerplate code less efficiently, >> it is still good to have that boilerplate defined once, instead of cutting >> and pasting. Or, at least, it is better *if* that once doesn't become >> unreadable in the process. > I am unable to assess the value of this mechanism unless you make a > concrete proposal. You seem to have something in mind but you're not > doing a good job getting it into mine... I'm not confident that macros are even a good idea; I just don't want a series of half-macros. That said, here is a strawman. defmacro boiler1(name, rejects): def %(name) (*args): for a in args: if a in %(rejects): print "Don't send me %s" % a ... boiler1(novowels, "aeiouy") boiler2(nokey5, "jkl") I'm pretty sure that a real version should accept suites instead of just arguments, and the variable portion might even be limited to suites (as in the thunk discussion). It might even be reasonable to mark macro calls as different from function calls. template novowels from boiler1("aeiou"): <suite> but I can't help thinking that multiple suites should be possible, and then they should be named, and ... that spurred at least one objection. http://mail.python.org/pipermail/python-dev/2005-April/052949.html -jJ

2 1

atexit missing an unregister method
by Nick Jacobson April 26, 2005

April 26, 2005

I was looking at the atexit module the other day; it seems like an elegant way to ensure that resources are cleaned up (that the garbage collector doesn't take care of). But while you can mark functions to be called with the 'register' method, there's no 'unregister' method to remove them from the stack of functions to be called. Nor is there any way to view this stack and e.g. call 'del' on a registered function. This would be useful in the following scenario, in which x and y are resources that need to be cleaned up, even in the event of a program exit: import atexit def free_resource(resource): ... atexit.register(free_resource, x) atexit.register(free_resource, y) # do operations with x and y, potentially causing the program to exit ... # if nothing caused the program to unexpectedly quit, close the resources free_resource(x) free_resource(y) #unregister the functions, so that you don't try to free the resources twice! atexit.unregisterall() Alternatively, it would be great if there were a way to view the stack of registered functions, and delete them from there. --Nick Jacobson _________________________________________________________________ Dont just search. Find. Check out the new MSN Search! http://search.msn.click-url.com/go/onm00200636ave/direct/01/

6 5

python.org crashing Mozilla?
by Paul Dubois April 26, 2005

April 26, 2005

Three different computers running Linux / Mozilla are crashing Mozilla when directed to python.org. A Netscape works ok. Are we hacked or are we showing off?

3 2

a few SF bugs which can (probably) be closed
by Ilya Sandler April 26, 2005

April 26, 2005

Good morning/evening/: Here a few sourceforge bugs which can probably be closed: [ 1168983 ] : ftplib.py string index out of range Original poster reports that the problem disappeared after a patch committed by Raymond [ 1178863 ] Variable.__init__ uses self.set(), blocking specialization seems like a dup of 1178872 [ 415492 ] Compiler generates relative filenames seems to have been fixed at some point. I could not reproduce it with python2.4 [ 751612 ] smtplib crashes Windows Kernal. Seems like an obvious Windows bug (not python's bug) and seems to be unreproducible Ilya

2 1

RE: [Python-Dev] Problem with embedded python
by Ugo Di Girolamo April 26, 2005

April 26, 2005

Sorry. will do. Ugo -----Original Message----- From: "Martin v. Löwis" [mailto:martin@v.loewis.de] Sent: Tuesday, April 26, 2005 2:31 PM To: Ugo Di Girolamo Cc: python-dev(a)python.org Subject: Re: [Python-Dev] Problem with embedded python Ugo Di Girolamo wrote: > What am I doing wrong? This is not the forum to ask this question, please use python-list(a)python.org instead. Regards, Martin

1 0

Problem with embedded python
by Ugo Di Girolamo April 26, 2005

April 26, 2005

I have the following code, that seems to make sense to me. However, it crashes about 1/3 of the times. My platform is Python 2.4.1 on WXP (I tried the release version from the msi and the debug version built by me, both downloaded today to have the latest version). The crash happens while the main thread is in Py_Finalize. I traced the crash to _Py_ForgetReference(op) in object.c at line 1847, where I have op->_ob_prev == NULL. What am I doing wrong? I'm definitely not too sure about the way I'm handling the GIL. Thanks in adv for any suggestion/ comment Cheers and ciao Ugo ////////////////////////// TestPyThreads.py ////////////////////////// #include <windows.h> #include "Python.h" int main() { PyEval_InitThreads(); Py_Initialize(); PyGILState_STATE main_restore_state = PyGILState_UNLOCKED; PyGILState_Release(main_restore_state); // start the thread { PyGILState_STATE state = PyGILState_Ensure(); int trash = PyRun_SimpleString( "import thread\n" "import time\n" "def foo():\n" " f = open('pippo.out', 'w', 0)\n" " i = 0;\n" " while 1:\n" " f.write('%d\\n'%i)\n" " time.sleep(0.01)\n" " i += 1\n" "t = thread.start_new_thread(foo, ())\n" ); PyGILState_Release(state); } // wait 300 ms Sleep(300); PyGILState_Ensure(); Py_Finalize(); return 0; }

2 1

Re: atexit missing an unregister method
by Nick Jacobson April 26, 2005

April 26, 2005

Raymond Hettinger wrote: << Will mull it over for a while. My first impression is that try/finally is a better tool for the scenario you outlined. >> You're right. try/finally takes care of my sample scenario. There may still be a case to be made for atexit.unregister(), though. --Nick Jacobson _________________________________________________________________ On the road to retirement? Check out MSN Life Events for advice on how to get there! http://lifeevents.msn.com/category.aspx?cid=Retirement

3 2