In a function you can use a return statement to break out of execution in the middle of the function. With modules you have no recourse. This is akin to return statements being allowed only at the end of a function. There are a small number of ways you can work around this, but they aren't great. This includes using wrapper modules or import hooks or sometimes from-import-*. Otherwise, if your module's execution is conditional, you end up indenting everything inside an if/else statement. Proposal: introduce a non-error mechanism to break out of module execution. This could be satisfied by a statement like break or return, though those specific ones could be confusing. It could also involve raising a special subclass of ImportError that the import machinery simply handles as not-an-error. This came up last year on python-list with mixed results. [1] However, time has not dimmed the appeal for me so I'm rebooting here. While the proposal seems relatively minor, the use cases are not extensive. <wink> The three main ones I've encountered are these: 1. C extension module with fallback to pure Python: try: from _extension_module import * except ImportError: pass else: break # or whatever color the bikeshed is # pure python implementation goes here 2. module imported under different name: if __name__ != "expected_name": from expected_name import * break # business as usual 3. module already imported under a different name: if "other_module" in sys.modules: exec("from other_module import *", globals()) break # module code here Thoughts? -eric [1] http://mail.python.org/pipermail/python-list/2011-June/1274424.html
On Tue, 24 Apr 2012 13:23:53 -0600 Eric Snow <ericsnowcurrently@gmail.com> wrote:
I think good practice should lead you to put your initialization code in a dedicated function that you call from your module toplevel. In this case, breaking out of execution is a matter of adding a return statement. I'm not sure the particular use cases you brought up are a good enough reason to add a syntactical construct. Regards Antoine.
Antoine Pitrou wrote:
True, but that doesn't prevent import from being run, functions and classes from being defined and resources being bound which are not going to get used. Think of code like this (let's assume the "break" statement is used for stopping module execution): """ # # MyModule # ### Try using the fast variant try: from MyModule_C_Extension import * except ImportError: pass else: # Stop execution of the module code object right here break ### Ah, well, so go ahead with the slow version import os, sys from MyOtherPackage import foo, bar, baz class MyClass: ... def MyFunc(a,b,c): ... def main(): ... if __name__ == '__main__': main() """ You can solve this by using two separate modules and a top-level module to switch between the implementations, but that's cumbersome if you have more than just a few of such modules in a package. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 24 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 4 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 24.04.2012 21:58, M.-A. Lemburg wrote:
What's wrong with an if statement on module level, if you even care about this?
There's a subtle bug here that shows that the proposed feature has its awkward points: you probably want to execute the "if __name__ == '__main__'" block in the C extension case as well. Georg
Georg Brandl wrote:
You'd have to indent the whole module. Been there, done that, doesn't look nice :-)
No, you don't :-) If you would have wanted that to happen, you'd put the "if __name__..." into the else: branch. You think of the "break" as having the same functionality as a "return" in a function. If reusing a statement is too much trouble, the same functionality could be had with an exception that get's caught by the executing (import) code. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 24 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 4 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
Georg Brandl wrote:
Ok, you got me :-) Should've paid more attention. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 24 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 4 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
M.-A. Lemburg wrote:
Georg Brandl wrote:
On 24.04.2012 22:20, M.-A. Lemburg wrote:
I think you have inadvertently demonstrated that that this proposed feature is hard to use correctly. Possibly even harder to use than existing idioms for solving the problems this is meant to solve. If the user does use it, they will likely need to duplicate code, which encourages copy-and-paste programming. Even if break at the module level is useful on rare occasions, I think the usefulness is far outweighed by the costs: - hard to use correctly, hence code using this feature risks being buggy - encourages premature micro-optimization, or at least the illusion of optimization - encourages or requires duplicate code and copy-and-paste programming - complicates the top-level program flow Today, if you successfully import a module, you know that all the top-level code in that module was executed. If this feature is added, you cannot be sure what top-level code was reached unless you scan through all the code above it. In my opinion, this is an attractive nuisance. -1 on the feature. -- Steven
On Wed, Apr 25, 2012 at 10:55 AM, Steven D'Aprano <steve@pearwood.info> wrote:
In my opinion, this is an attractive nuisance.
-1 on the feature.
Agreed (and my preferred idiom for all the cited cases is also "always define the Python version, override at the end with the accelerated version"). Although, if we *did* do it, I think allowing "return" at module level would be the way to proceed (as Mark Shannon noted, that's only *disallowed* now because the compiler specifically prevents it. The eval loop itself understands it just fine and ceases execution as soon as it encounters the relevant bytecode. It isn't quite as simple as just deleting those lines though, since we likely still wouldn't want to allow return statements in class bodies). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 04/25/2012 02:58 AM, Greg Ewing wrote:
Currently return isn't allowed in class bodies defined inside functons. So it probably won't work in top level either.
As far as the feature goes, it wouldn't be consistent with class behaviour unless you allow the returns in class's to work too. Think of modules as a type of class where ... import module is equivalent to ... module module_name: <module file contents here> Like classes the module body would execute to define the module, and return inside the module body would be a syntax error. Of course since modules aren't specifically defined in this way, there is the option to not follow that consultancy. Cheers, Ron Ron
On 04/25/2012 02:31 PM, Georg Brandl wrote:
Not quite the same thing, but I see how you would think that from the way I wrote the example. I didn't mean the file to be inserted, but instead as if it was written in the module statement body. That is, if we even had a "module" keyword, which we don't. ;-) The point is that a module contents execute from beginning to end to create a module, in the same way a class's contents execute from beginning to end to create a class. There are also differences, such as where a module is stored and how it's contents are accessed, and so they are not the same thing. You can't just change a class into a module and vise-versa by just changing it's header or moving it's body into a separate file. I was just trying to point out a module is closer to a class than it is to a function, and that is a good thing. Allowing a return or break in a module could make things more confusing. Also, by not allowing return or breaks, it catches errors were the indentation is lost in functions or methods quicker. Cheers, Ron
On 4/25/2012 4:27 PM, Ron Adam wrote:
I was just trying to point out a module is closer to a class than it is to a function, and that is a good thing.
Each should only be executed once, and this is enforced for modules by the sys.modules cache.
Allowing a return or break in a module could make things more confusing.
Agreed. It is not actually necessary for functions to have an explicit return statement. I believe that there are languages that define the function return value as the last value assigned to the function name. def fact(n): if n > 1: fact = n*fact(n-1) else: fact = 1 (This is pretty close to how mathematicians might write the definition. It does, however, require special-casing the function name on the left of '='.) 'return' is, however, useful for returning out of more deeply nested constructs, especially loops, without setting flag variables or having multi-level breaks. This consideration does not really apply to the use case for module return. A virtue of defining everything in Python and then trying to import the accelerated override is that different implementations and versions thereof can use the same file even though they accelerate different parts and amounts (even none) of the module. -- Terry Jan Reedy
Nick Coghlan wrote:
IMO, defining things twice in the same module is not a very Pythonic way of designing Python software. Left aside the resource leakage, it also makes if difficult to find the implementation that actually gets used, bypasses "explicit is better than implicit", and it doesn't address possible side-effects of the definitions that you eventually override at the end of the module. Python is normally written with a top-to-bottom view in mind, where you don't expect things to suddenly change near the end. This is why we introduced decorators before the function definition, rather than place them after the function definition. It's also why we tend to put imports, globals, helpers at the top of the file. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 25 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 3 days to go 2012-04-25: Released eGenix mx Base 3.2.4 http://egenix.com/go27 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Wed, Apr 25, 2012 at 7:41 PM, M.-A. Lemburg <mal@egenix.com> wrote:
I agree overwriting at the end isn't ideal, but I don't think allowing returns at module level is a significant improvement. I'd rather see a higher level approach that specifically set out to tackle the problem of choosing between multiple implementations of a module at runtime that cleanly supported *testing* all the implementations in a single process, while still having one implementation that was used be default. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan wrote:
Isn't that an application developer choice to make rather than one that we force upon the developer and one which only addresses a single use case (having multiple implementation variants in a module) ? What about other use cases, where you e.g. * know that the subsequent function/class definitions are going to fail, because your runtime environment doesn't provide the needed functionality ? * want to limit the available defined APIs based on flags or other settings ? * want to make modules behave more like functions or classes ? * want to debug import loops ? Since the module body is run more or less like a function or class body, it seems natural to allow the same statements available there in modules as well. Esp. with the new importlib, tapping into the wealth of functionality in that area has become a lot easier than before. Only the compiler is preventing it. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 25 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 3 days to go 2012-04-25: Released eGenix mx Base 3.2.4 http://egenix.com/go27 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
M.-A. Lemburg dixit (2012-04-25, 12:37):
IMHO then you should refactor your module into a few smaller ones.
* want to limit the available defined APIs based on flags or other settings ?
As above if there are many and/or long variants. Otherwise top-level if/else should be sufficient.
* want to make modules behave more like functions or classes ?
What for? Use functions or classes then.
* want to debug import loops ?
Don't we have some good ways and tools to do it anyway?
Function body and class body are completely different stories. * The former can contain return, the latter not (and IMHO this limitation is a good thing -- see below...). * The former is executed many times (each time creating a new local namespace); the latter is executed immediately and only once. * The former takes some input (arguments) and returns some output; the latter is a container for some callables and/or some data. * The former is mostly relatively small; the latter is often quite large (breaking execution flow of large bodies is often described as a bad practice, and not without good reasons). Modules resomehow similar to singleton classes (instantiated with the first import). Cheers. *j
On Wed, Apr 25, 2012 at 5:52 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
... tackle the problem of choosing between multiple implementations of a module at runtime ...
I like the idea, but I'm not sure I would like the result. The obvious path (at least for experimenting) is to replace the import statement with an import function. (a) At the interactive prompt, the function would return the module, and therefore allow it to be referenced as _. (I don't always remember the "as shortname" until I've already hit enter. And often all I really want is to see help(module), but without switching to another window.) (b) The functional interface could expose a configuration object, so that in addition to deciding between alternate implementations, a single implementation could set up objects differently. (Do I really need to define that type of handler? What loggers should I enable initially, even while setting up the rest of the logging machinery? OK, let me open that database connection before I define this class.) These are both features I have often wanted. But I don't want to deal with the resulting questions, like "Wait, how can that logger not exist? Oh, someone else imported logging first..." And I'm not sure it is possible to get the freedom of (b) without those problems, unless modules stop being singletons. -jJ
M.-A. Lemburg wrote:
You're not defining things twice in the same module. You're designing them twice in two modules, one in Python and one in a C extension module. Your own example does the same thing. The only difference is that you try to avoid creating the pure-Python versions if you don't need them, but you still have the source code for them in the module.
Left aside the resource leakage,
What resource leakage? If I do this: def f(): pass from module import f then the first function f is garbage collected and there is no resource leakage. As for the rest of your post, I'm afraid that I find most of it "not even wrong" so I won't address it directly. I will say this: I'm sure you can come up with all sorts of reasons for not liking the current idiom of "define pure-Python code first, then replace with accelerated C version if available", e.g. extremely unlikely scenarios for code that has side-effects that you might want to avoid. That's all fine. The argument is not that there is never a use for a top level return, or that alternatives are perfect. The argument is that a top level return has more disadvantages than advantages. Unless you address the disadvantages and costs of top level return, you won't convince me, and I doubt you will convince many others. -- Steven
Mark Shannon wrote:
Quoting from my post earlier today: [quote] Even if break at the module level is useful on rare occasions, I think the usefulness is far outweighed by the costs: - hard to use correctly, hence code using this feature risks being buggy - encourages premature micro-optimization, or at least the illusion of optimization - encourages or requires duplicate code and copy-and-paste programming - complicates the top-level program flow Today, if you successfully import a module, you know that all the top-level code in that module was executed. If this feature is added, you cannot be sure what top-level code was reached unless you scan through all the code above it. [end quote] And to see the context: http://mail.python.org/pipermail/python-ideas/2012-April/014897.html -- Steven
Steven D'Aprano wrote:
I would have taken these to be the disadvantages, rather costs. By costs, I assumed you meant implementation effort or runtime overhead. Also, I don't see the difference between a return in a module, and a return in a function. Both terminate execution and return to the caller. Why do these four points apply to module-level returns any more than function-level returns?
I don't know if this is a good idea or not, but the fact that to it can be implemented by removing a single restriction in the compiler suggests it might have some merit. Cheers, Mark.
Mark Shannon wrote: [...]
I would have taken these to be the disadvantages, rather costs. By costs, I assumed you meant implementation effort or runtime overhead.
No, costs as in "costs versus benefits".
A very good point. There is a school of thought that functions should always have a single entry (the top of the function) and a single exit (the bottom). If I recall correctly, Pascal is like that: there's no way to return early from a function in Pascal. However, code inside a function normally performs a calculation and returns a value, so once that value is calculated there's no point hanging around. The benefit of early return in functions outweighs the cost. It is my argument that this is not the case for top level module code. Some differences between return in a function and return in a module: 1) Modules don't have a caller as such, so it isn't clear what you are returning too. (If the module is being imported, I suppose you could call the importing module the caller; but when the module is being run instead, there is no importing module.) So a top level return is more like an exit than a return, except it doesn't actually exit the Python interpreter. 2) When functions unexpectedly return early, you can sometimes get a clue why by inspecting the return value. Modules don't have a return value. 3) Functions tend to be relatively small (or at least, they should be relatively small), so while an early return in the middle of a function can be surprising, the cost of discovering that is not very high. In contrast, modules tend to be relatively large, hundreds or even thousands of lines. An early return could be anywhere. 4) Code at the top level of modules is usually transparent: the details of what gets done are important. People will want to know which functions, classes and global variables are actually created, and which are skipped due to an early return. In contrast, functions are usually treated as opaque blackboxes: people usually care about the interface, not the implementation. So typically they don't care whether the function returns out early or not. There may be other differences.
Do you really mean to say that *because* something is easy, it therefore might be a good idea? rm -rf / Easy, and therefore a good idea, yes? *wink* -- Steven
On Thu, 26 Apr 2012 02:54:58 +1000 Steven D'Aprano <steve@pearwood.info> wrote:
I read it as an expression of the language design philosophy that the best way to add power is to remove restrictions. Personally, I agree with that philosophy, as removing a single restriction is a much better alternative than having a flock of tools, syntax and special cases to compensate. Compare Python - where functions are first-class objects and can be trivially passed as arguments - to pretty much any modern language that restricts such usage. That said, "more power" is not always the best choice from a language design point of few. In this case there's really only one use case for lifting the restriction against return in classes and modules, and the problems already pointed out that lifting this restriction creates outweigh the benefits of that use case. -1. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
If this is to be done I'd like to see all special methods supported. One of particular interest to modules is __getattr__... I think the idea is crazy and will lead to chicken and egg discussions.
I'm actually fine with the way things are and would _really_ have used a module "break" perhaps once. As I alluded to originally, there just aren't enough good use cases to make it worth it over the status quo. [1][2] At the very least, the mailing list archives will have a pretty good discussion on the idea (of which I could not find one previously). To recap, this idea is about making the intent/context of a module explicit at the beginning, rather than the end -- without resorting to the extra level(s) of indent that an if/else solution would require. Decorators and the with statement (both targeting code blocks) came about for the same reason. However, a simple return/break statement would allow much more than that. As Nick suggested, a more specific, targeted solution would be better. ("import-else" doesn't fit the bill. [3]) For the record, I still think the status quo is sub-optimal. My original post lists what I think are legitimate (if uncommon) use cases. Here's my list of (nitpick-ish?) concerns with the current solutions: 1. if/else to make context explicit: one extra level of indent (or ever-so-slightly-possibly more) 2. conditionally replace module contents at the end: without a clear comment at the beginning, may miscommunicate the final contents of the module 3. put the code in the else of #1 in a separate module: one more import involved (weak, I know), and one more level of FS indirection ("flat is better than nested") 4. special exception + import hook: not worth the trouble -eric [1] http://www.boredomandlaziness.org/2011/02/justifying-python-language-changes... [2] http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html [3] An idea that has come up before (and at least once recently): import cdecimal else decimal as decimal <==> try: import cdecimal as decimal except ImportError: import decimal
On Wed, Apr 25, 2012 at 6:30 AM, Steven D'Aprano <steve@pearwood.info> wrote:
This got me thinking "well, you get the same thing with functions and the return statement". Then I realized there's a problem with that line of thinking and stepped back. Modules and functions have distinct purposes (by design) and we shouldn't help make that distinction blurrier. We should (and mostly do) teach the concept of a module as a top-level (singleton) namespace definition. The idioms presented in this thread mostly bear this out. Python doesn't force the distinction syntactically, nor am I suggesting it should. However, it seems to me that this not how most people think of modules. The culprit here is the lack of distinction between modules and scripts. If a module is like a class, a script is like a function. Perhaps we should consider ways of making the difference between scripts and modules clearer, whether in the docs, with syntax, or otherwise. -eric
On 25 Apr, 2012, at 14:11, Mark Shannon wrote:
Harder to understand code is one disadvantage. The "return" that ends execution can easily be hidden in a list of definitions, such as ... some definitions ... if sys.platform != 'win32': return ... more definitions for win32 specific functionality ... That's easy to read with a 10 line module, but not when the module gets significantly larger. Also, why use the proposed module-scope return instead of an if-statement with nested definitions, this works just fine: : def foo(): pass : : if sys.platform == 'linux': : : def linux_bar(): pass
Ronald Oussoren wrote:
Because this only works reasonably if you have a few lines of code to indent. As soon as you have hundreds of lines, this becomes both unreadable and difficult to edit. The above is how the thread was started, BTW :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 25 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 3 days to go 2012-04-25: Released eGenix mx Base 3.2.4 http://egenix.com/go27 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
(sent from my phone) On Apr 25, 2012 3:01 PM, "M.-A. Lemburg" <mal@egenix.com> wrote:
Ronald Oussoren wrote:
Also, why use the proposed module-scope return instead of an
if-statement with nested definitions, this works just fine:
OTOH the return statement becomes really hard to spot... Arnaud
Arnaud Delobelle wrote:
People don't appear to have a problem with this in long functions or methods, so I'm not sure how well that argument qualifies. The programmer can of course add an easy to spot comment where the return is used. Just as question of programming style. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 25 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 3 days to go 2012-04-25: Released eGenix mx Base 3.2.4 http://egenix.com/go27 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
M.-A. Lemburg wrote:
People don't appear to have a problem with this in long functions or methods, so I'm not sure how well that argument qualifies.
Well, I have a problem with long functions whether they have embedded returns or not. But another difference is that functions usually consist of a set of related statements that need to be read in order from top to bottom. Modules, on the other hand, typically consist of multiple relatively small units whose order mostly doesn't matter. Sticking a return in the middle introduces an unexpected ordering dependency. -- Greg
On 25.04.2012 16:00, M.-A. Lemburg wrote:
So you don't have any classes that span hundreds of lines? I don't see how this is different in terms of editing difficulty. As for readability, at least you can see from the indentation that there's something special about the module code in question. You don't see that if there's a "return" scattered somewhere. cheers, Georg PS: I don't buy the "it's no problem with functions" argument. Even though I'm certainly guilty of writing 100+ line functions, I find them quite ungraspable (is that a word?) and usually try to limit functions and methods to a screenful.
On Wed, Apr 25, 2012 at 5:41 AM, M.-A. Lemburg <mal@egenix.com> wrote:
Nick Coghlan wrote:
On Wed, Apr 25, 2012 at 10:55 AM, Steven D'Aprano <steve@pearwood.info> wrote:
IMO, defining things twice in the same module is not a very Pythonic way of designing Python software.
Agreed, which is one problem with accelerator modules.
Left aside the resource leakage, it also makes if difficult to find the implementation that actually gets used,
Agreed, which is another problem with accelerator modules.
bypasses "explicit is better than implicit",
Agreed, which is is another problem with import * from accelerator So far, these reasons have been saying "Don't do that if you can help it", and if you do it anyhow, making the code stick out is better than nothing. (That said, making it stick out by having top-level definitions that are *not* at the far left is ... unfortunate.)
That is a real problem, but the answer is to be more explicit. try: from _foo import A except ImportError ... # define exactly A, as it will be needed or try: A except NameError: ... # define exactly A, as it will be needed or even create a wrapper that imports or defines your object the first time it is called...
Python is normally written with a top-to-bottom view in mind, where you don't expect things to suddenly change near the end.
Actually, I tend to (wrongly) view the module-level code as fixed declarations rather than commands, so that leaving a name undefined or defining it conditionally is just as bad. Having an alternative definition in the same location is less of a problem. I have learned to look for import * at the beginning and end of a module, but more special cases would not be helpful. -jJ
On Tue, Apr 24, 2012 at 8:55 PM, Steven D'Aprano <steve@pearwood.info> wrote:
- encourages or requires duplicate code and copy-and-paste programming
How does stopping a module definition early encourage duplicate code? Just because you might have to react differently after a full import vs a partial?
Today, if you successfully import a module, you know that all the top-level code in that module was executed.
No, you don't. On the other hand, the counterexamples (circular imports, some lazy imports, module-name clashes, top-level definitions dependent on what was already imported by other modules, etc) are already painful to debug. -jJ
On 04/24/2012 03:15 PM, Mark Shannon wrote:
Does it also allow returns in class bodies when you do that?
Weather or not it does. I think modules are closer to class bodies and return should be a syntax error in that case as well. They aren't functions and we shouldn't think of them that way. IMHO It would make Python harder to learn when the lines between them get blurred. Cheers, Ron
M.-A. Lemburg wrote:
Antoine Pitrou wrote:
Premature micro-optimizations. Just starting the interpreter runs imports, defines functions and classes, and binds resources which your script may never use. Lists and dicts are over-allocated, which you may never need. Ints and strings are boxed. Python is a language that takes Knuth seriously: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." In the majority of cases, a few more small inefficiencies, especially those that are one-off costs at startup, are not a big deal. In those cases where it is a big deal, you can place code inside if-blocks, factor it out into separate modules, or use delayed execution by putting it inside functions. It may not be quite so pretty, but it gets the job done, and it requires no new features.
You can also solve it by defining the slow version first, as a fall-back, then replace it with the fast version, if it exists: import os, sys from MyOtherPackage import foo, bar, baz class MyClass: ... def MyFunc(a,b,c): ... def main(): ... try: from MyModule_C_Extension import * except ImportError: pass if __name__ == '__main__': main() The advantage of this is that MyModule_C_Extension doesn't need to supply everything or nothing. It may only supply (say) MyClass, and MyFunc will naturally fall back on the pre-defined Python versions. This is the idiom used by at least the bisect and pickle modules, and I think it is the Pythonic way. -- Steven
On 24 April 2012 20:23, Eric Snow <ericsnowcurrently@gmail.com> wrote:
For what it's worth I've wanted this a couple of times. There are always workarounds of course (but not particularly pretty sometimes). Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On 24Apr2012 13:23, Eric Snow <ericsnowcurrently@gmail.com> wrote: | In a function you can use a return statement to break out of execution | in the middle of the function. With modules you have no recourse. | This is akin to return statements being allowed only at the end of a | function. [...] Something very similar came up some months ago I think. My personal suggestion was: raise StopImport to cleanly exit a module without causing an ImportError, just as exiting a generator raises Stopiteration in the caller. In this case the caller is the import machinery, which should consider this except a clean completion of the import. Cheers, -- Cameron Simpson <cs@zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ Disclaimer: Opinions expressed here are CORRECT, mine, and not PSLs or NMSUs.. - Larry Cunningham <larry@psl.nmsu.edu>
Eric Snow, 24.04.2012 21:23:
Without having read through the thread, I think that code that needs this is just badly structured. All of the above cases can be fixed by moving the code into a separate (and appropriately named) module and importing conditionally from that. I'm generally -1 on anything that would allow non-error code at an arbitrary place further up in a module to prevent the non-indented module code I'm looking at from being executed. Whether the result of that execution makes it into the module API or not is a different question that is commonly answered either by "__all__" at the very top of a module or by the code at the very end of the module, not in between. Stefan
On Tue, 24 Apr 2012 13:23:53 -0600 Eric Snow <ericsnowcurrently@gmail.com> wrote:
I think good practice should lead you to put your initialization code in a dedicated function that you call from your module toplevel. In this case, breaking out of execution is a matter of adding a return statement. I'm not sure the particular use cases you brought up are a good enough reason to add a syntactical construct. Regards Antoine.
Antoine Pitrou wrote:
True, but that doesn't prevent import from being run, functions and classes from being defined and resources being bound which are not going to get used. Think of code like this (let's assume the "break" statement is used for stopping module execution): """ # # MyModule # ### Try using the fast variant try: from MyModule_C_Extension import * except ImportError: pass else: # Stop execution of the module code object right here break ### Ah, well, so go ahead with the slow version import os, sys from MyOtherPackage import foo, bar, baz class MyClass: ... def MyFunc(a,b,c): ... def main(): ... if __name__ == '__main__': main() """ You can solve this by using two separate modules and a top-level module to switch between the implementations, but that's cumbersome if you have more than just a few of such modules in a package. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 24 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 4 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 24.04.2012 21:58, M.-A. Lemburg wrote:
What's wrong with an if statement on module level, if you even care about this?
There's a subtle bug here that shows that the proposed feature has its awkward points: you probably want to execute the "if __name__ == '__main__'" block in the C extension case as well. Georg
Georg Brandl wrote:
You'd have to indent the whole module. Been there, done that, doesn't look nice :-)
No, you don't :-) If you would have wanted that to happen, you'd put the "if __name__..." into the else: branch. You think of the "break" as having the same functionality as a "return" in a function. If reusing a statement is too much trouble, the same functionality could be had with an exception that get's caught by the executing (import) code. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 24 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 4 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
Georg Brandl wrote:
Ok, you got me :-) Should've paid more attention. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 24 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 4 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
M.-A. Lemburg wrote:
Georg Brandl wrote:
On 24.04.2012 22:20, M.-A. Lemburg wrote:
I think you have inadvertently demonstrated that that this proposed feature is hard to use correctly. Possibly even harder to use than existing idioms for solving the problems this is meant to solve. If the user does use it, they will likely need to duplicate code, which encourages copy-and-paste programming. Even if break at the module level is useful on rare occasions, I think the usefulness is far outweighed by the costs: - hard to use correctly, hence code using this feature risks being buggy - encourages premature micro-optimization, or at least the illusion of optimization - encourages or requires duplicate code and copy-and-paste programming - complicates the top-level program flow Today, if you successfully import a module, you know that all the top-level code in that module was executed. If this feature is added, you cannot be sure what top-level code was reached unless you scan through all the code above it. In my opinion, this is an attractive nuisance. -1 on the feature. -- Steven
On Wed, Apr 25, 2012 at 10:55 AM, Steven D'Aprano <steve@pearwood.info> wrote:
In my opinion, this is an attractive nuisance.
-1 on the feature.
Agreed (and my preferred idiom for all the cited cases is also "always define the Python version, override at the end with the accelerated version"). Although, if we *did* do it, I think allowing "return" at module level would be the way to proceed (as Mark Shannon noted, that's only *disallowed* now because the compiler specifically prevents it. The eval loop itself understands it just fine and ceases execution as soon as it encounters the relevant bytecode. It isn't quite as simple as just deleting those lines though, since we likely still wouldn't want to allow return statements in class bodies). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 04/25/2012 02:58 AM, Greg Ewing wrote:
Currently return isn't allowed in class bodies defined inside functons. So it probably won't work in top level either.
As far as the feature goes, it wouldn't be consistent with class behaviour unless you allow the returns in class's to work too. Think of modules as a type of class where ... import module is equivalent to ... module module_name: <module file contents here> Like classes the module body would execute to define the module, and return inside the module body would be a syntax error. Of course since modules aren't specifically defined in this way, there is the option to not follow that consultancy. Cheers, Ron Ron
On 04/25/2012 02:31 PM, Georg Brandl wrote:
Not quite the same thing, but I see how you would think that from the way I wrote the example. I didn't mean the file to be inserted, but instead as if it was written in the module statement body. That is, if we even had a "module" keyword, which we don't. ;-) The point is that a module contents execute from beginning to end to create a module, in the same way a class's contents execute from beginning to end to create a class. There are also differences, such as where a module is stored and how it's contents are accessed, and so they are not the same thing. You can't just change a class into a module and vise-versa by just changing it's header or moving it's body into a separate file. I was just trying to point out a module is closer to a class than it is to a function, and that is a good thing. Allowing a return or break in a module could make things more confusing. Also, by not allowing return or breaks, it catches errors were the indentation is lost in functions or methods quicker. Cheers, Ron
On 4/25/2012 4:27 PM, Ron Adam wrote:
I was just trying to point out a module is closer to a class than it is to a function, and that is a good thing.
Each should only be executed once, and this is enforced for modules by the sys.modules cache.
Allowing a return or break in a module could make things more confusing.
Agreed. It is not actually necessary for functions to have an explicit return statement. I believe that there are languages that define the function return value as the last value assigned to the function name. def fact(n): if n > 1: fact = n*fact(n-1) else: fact = 1 (This is pretty close to how mathematicians might write the definition. It does, however, require special-casing the function name on the left of '='.) 'return' is, however, useful for returning out of more deeply nested constructs, especially loops, without setting flag variables or having multi-level breaks. This consideration does not really apply to the use case for module return. A virtue of defining everything in Python and then trying to import the accelerated override is that different implementations and versions thereof can use the same file even though they accelerate different parts and amounts (even none) of the module. -- Terry Jan Reedy
Nick Coghlan wrote:
IMO, defining things twice in the same module is not a very Pythonic way of designing Python software. Left aside the resource leakage, it also makes if difficult to find the implementation that actually gets used, bypasses "explicit is better than implicit", and it doesn't address possible side-effects of the definitions that you eventually override at the end of the module. Python is normally written with a top-to-bottom view in mind, where you don't expect things to suddenly change near the end. This is why we introduced decorators before the function definition, rather than place them after the function definition. It's also why we tend to put imports, globals, helpers at the top of the file. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 25 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 3 days to go 2012-04-25: Released eGenix mx Base 3.2.4 http://egenix.com/go27 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Wed, Apr 25, 2012 at 7:41 PM, M.-A. Lemburg <mal@egenix.com> wrote:
I agree overwriting at the end isn't ideal, but I don't think allowing returns at module level is a significant improvement. I'd rather see a higher level approach that specifically set out to tackle the problem of choosing between multiple implementations of a module at runtime that cleanly supported *testing* all the implementations in a single process, while still having one implementation that was used be default. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan wrote:
Isn't that an application developer choice to make rather than one that we force upon the developer and one which only addresses a single use case (having multiple implementation variants in a module) ? What about other use cases, where you e.g. * know that the subsequent function/class definitions are going to fail, because your runtime environment doesn't provide the needed functionality ? * want to limit the available defined APIs based on flags or other settings ? * want to make modules behave more like functions or classes ? * want to debug import loops ? Since the module body is run more or less like a function or class body, it seems natural to allow the same statements available there in modules as well. Esp. with the new importlib, tapping into the wealth of functionality in that area has become a lot easier than before. Only the compiler is preventing it. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 25 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 3 days to go 2012-04-25: Released eGenix mx Base 3.2.4 http://egenix.com/go27 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
M.-A. Lemburg dixit (2012-04-25, 12:37):
IMHO then you should refactor your module into a few smaller ones.
* want to limit the available defined APIs based on flags or other settings ?
As above if there are many and/or long variants. Otherwise top-level if/else should be sufficient.
* want to make modules behave more like functions or classes ?
What for? Use functions or classes then.
* want to debug import loops ?
Don't we have some good ways and tools to do it anyway?
Function body and class body are completely different stories. * The former can contain return, the latter not (and IMHO this limitation is a good thing -- see below...). * The former is executed many times (each time creating a new local namespace); the latter is executed immediately and only once. * The former takes some input (arguments) and returns some output; the latter is a container for some callables and/or some data. * The former is mostly relatively small; the latter is often quite large (breaking execution flow of large bodies is often described as a bad practice, and not without good reasons). Modules resomehow similar to singleton classes (instantiated with the first import). Cheers. *j
On Wed, Apr 25, 2012 at 5:52 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
... tackle the problem of choosing between multiple implementations of a module at runtime ...
I like the idea, but I'm not sure I would like the result. The obvious path (at least for experimenting) is to replace the import statement with an import function. (a) At the interactive prompt, the function would return the module, and therefore allow it to be referenced as _. (I don't always remember the "as shortname" until I've already hit enter. And often all I really want is to see help(module), but without switching to another window.) (b) The functional interface could expose a configuration object, so that in addition to deciding between alternate implementations, a single implementation could set up objects differently. (Do I really need to define that type of handler? What loggers should I enable initially, even while setting up the rest of the logging machinery? OK, let me open that database connection before I define this class.) These are both features I have often wanted. But I don't want to deal with the resulting questions, like "Wait, how can that logger not exist? Oh, someone else imported logging first..." And I'm not sure it is possible to get the freedom of (b) without those problems, unless modules stop being singletons. -jJ
M.-A. Lemburg wrote:
You're not defining things twice in the same module. You're designing them twice in two modules, one in Python and one in a C extension module. Your own example does the same thing. The only difference is that you try to avoid creating the pure-Python versions if you don't need them, but you still have the source code for them in the module.
Left aside the resource leakage,
What resource leakage? If I do this: def f(): pass from module import f then the first function f is garbage collected and there is no resource leakage. As for the rest of your post, I'm afraid that I find most of it "not even wrong" so I won't address it directly. I will say this: I'm sure you can come up with all sorts of reasons for not liking the current idiom of "define pure-Python code first, then replace with accelerated C version if available", e.g. extremely unlikely scenarios for code that has side-effects that you might want to avoid. That's all fine. The argument is not that there is never a use for a top level return, or that alternatives are perfect. The argument is that a top level return has more disadvantages than advantages. Unless you address the disadvantages and costs of top level return, you won't convince me, and I doubt you will convince many others. -- Steven
Mark Shannon wrote:
Quoting from my post earlier today: [quote] Even if break at the module level is useful on rare occasions, I think the usefulness is far outweighed by the costs: - hard to use correctly, hence code using this feature risks being buggy - encourages premature micro-optimization, or at least the illusion of optimization - encourages or requires duplicate code and copy-and-paste programming - complicates the top-level program flow Today, if you successfully import a module, you know that all the top-level code in that module was executed. If this feature is added, you cannot be sure what top-level code was reached unless you scan through all the code above it. [end quote] And to see the context: http://mail.python.org/pipermail/python-ideas/2012-April/014897.html -- Steven
Steven D'Aprano wrote:
I would have taken these to be the disadvantages, rather costs. By costs, I assumed you meant implementation effort or runtime overhead. Also, I don't see the difference between a return in a module, and a return in a function. Both terminate execution and return to the caller. Why do these four points apply to module-level returns any more than function-level returns?
I don't know if this is a good idea or not, but the fact that to it can be implemented by removing a single restriction in the compiler suggests it might have some merit. Cheers, Mark.
Mark Shannon wrote: [...]
I would have taken these to be the disadvantages, rather costs. By costs, I assumed you meant implementation effort or runtime overhead.
No, costs as in "costs versus benefits".
A very good point. There is a school of thought that functions should always have a single entry (the top of the function) and a single exit (the bottom). If I recall correctly, Pascal is like that: there's no way to return early from a function in Pascal. However, code inside a function normally performs a calculation and returns a value, so once that value is calculated there's no point hanging around. The benefit of early return in functions outweighs the cost. It is my argument that this is not the case for top level module code. Some differences between return in a function and return in a module: 1) Modules don't have a caller as such, so it isn't clear what you are returning too. (If the module is being imported, I suppose you could call the importing module the caller; but when the module is being run instead, there is no importing module.) So a top level return is more like an exit than a return, except it doesn't actually exit the Python interpreter. 2) When functions unexpectedly return early, you can sometimes get a clue why by inspecting the return value. Modules don't have a return value. 3) Functions tend to be relatively small (or at least, they should be relatively small), so while an early return in the middle of a function can be surprising, the cost of discovering that is not very high. In contrast, modules tend to be relatively large, hundreds or even thousands of lines. An early return could be anywhere. 4) Code at the top level of modules is usually transparent: the details of what gets done are important. People will want to know which functions, classes and global variables are actually created, and which are skipped due to an early return. In contrast, functions are usually treated as opaque blackboxes: people usually care about the interface, not the implementation. So typically they don't care whether the function returns out early or not. There may be other differences.
Do you really mean to say that *because* something is easy, it therefore might be a good idea? rm -rf / Easy, and therefore a good idea, yes? *wink* -- Steven
On Thu, 26 Apr 2012 02:54:58 +1000 Steven D'Aprano <steve@pearwood.info> wrote:
I read it as an expression of the language design philosophy that the best way to add power is to remove restrictions. Personally, I agree with that philosophy, as removing a single restriction is a much better alternative than having a flock of tools, syntax and special cases to compensate. Compare Python - where functions are first-class objects and can be trivially passed as arguments - to pretty much any modern language that restricts such usage. That said, "more power" is not always the best choice from a language design point of few. In this case there's really only one use case for lifting the restriction against return in classes and modules, and the problems already pointed out that lifting this restriction creates outweigh the benefits of that use case. -1. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/ Independent Software developer/SCM consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
If this is to be done I'd like to see all special methods supported. One of particular interest to modules is __getattr__... I think the idea is crazy and will lead to chicken and egg discussions.
I'm actually fine with the way things are and would _really_ have used a module "break" perhaps once. As I alluded to originally, there just aren't enough good use cases to make it worth it over the status quo. [1][2] At the very least, the mailing list archives will have a pretty good discussion on the idea (of which I could not find one previously). To recap, this idea is about making the intent/context of a module explicit at the beginning, rather than the end -- without resorting to the extra level(s) of indent that an if/else solution would require. Decorators and the with statement (both targeting code blocks) came about for the same reason. However, a simple return/break statement would allow much more than that. As Nick suggested, a more specific, targeted solution would be better. ("import-else" doesn't fit the bill. [3]) For the record, I still think the status quo is sub-optimal. My original post lists what I think are legitimate (if uncommon) use cases. Here's my list of (nitpick-ish?) concerns with the current solutions: 1. if/else to make context explicit: one extra level of indent (or ever-so-slightly-possibly more) 2. conditionally replace module contents at the end: without a clear comment at the beginning, may miscommunicate the final contents of the module 3. put the code in the else of #1 in a separate module: one more import involved (weak, I know), and one more level of FS indirection ("flat is better than nested") 4. special exception + import hook: not worth the trouble -eric [1] http://www.boredomandlaziness.org/2011/02/justifying-python-language-changes... [2] http://www.boredomandlaziness.org/2011/02/status-quo-wins-stalemate.html [3] An idea that has come up before (and at least once recently): import cdecimal else decimal as decimal <==> try: import cdecimal as decimal except ImportError: import decimal
On Wed, Apr 25, 2012 at 6:30 AM, Steven D'Aprano <steve@pearwood.info> wrote:
This got me thinking "well, you get the same thing with functions and the return statement". Then I realized there's a problem with that line of thinking and stepped back. Modules and functions have distinct purposes (by design) and we shouldn't help make that distinction blurrier. We should (and mostly do) teach the concept of a module as a top-level (singleton) namespace definition. The idioms presented in this thread mostly bear this out. Python doesn't force the distinction syntactically, nor am I suggesting it should. However, it seems to me that this not how most people think of modules. The culprit here is the lack of distinction between modules and scripts. If a module is like a class, a script is like a function. Perhaps we should consider ways of making the difference between scripts and modules clearer, whether in the docs, with syntax, or otherwise. -eric
On 25 Apr, 2012, at 14:11, Mark Shannon wrote:
Harder to understand code is one disadvantage. The "return" that ends execution can easily be hidden in a list of definitions, such as ... some definitions ... if sys.platform != 'win32': return ... more definitions for win32 specific functionality ... That's easy to read with a 10 line module, but not when the module gets significantly larger. Also, why use the proposed module-scope return instead of an if-statement with nested definitions, this works just fine: : def foo(): pass : : if sys.platform == 'linux': : : def linux_bar(): pass
Ronald Oussoren wrote:
Because this only works reasonably if you have a few lines of code to indent. As soon as you have hundreds of lines, this becomes both unreadable and difficult to edit. The above is how the thread was started, BTW :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 25 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 3 days to go 2012-04-25: Released eGenix mx Base 3.2.4 http://egenix.com/go27 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
(sent from my phone) On Apr 25, 2012 3:01 PM, "M.-A. Lemburg" <mal@egenix.com> wrote:
Ronald Oussoren wrote:
Also, why use the proposed module-scope return instead of an
if-statement with nested definitions, this works just fine:
OTOH the return statement becomes really hard to spot... Arnaud
Arnaud Delobelle wrote:
People don't appear to have a problem with this in long functions or methods, so I'm not sure how well that argument qualifies. The programmer can of course add an easy to spot comment where the return is used. Just as question of programming style. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 25 2012)
2012-04-28: PythonCamp 2012, Cologne, Germany 3 days to go 2012-04-25: Released eGenix mx Base 3.2.4 http://egenix.com/go27 ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
M.-A. Lemburg wrote:
People don't appear to have a problem with this in long functions or methods, so I'm not sure how well that argument qualifies.
Well, I have a problem with long functions whether they have embedded returns or not. But another difference is that functions usually consist of a set of related statements that need to be read in order from top to bottom. Modules, on the other hand, typically consist of multiple relatively small units whose order mostly doesn't matter. Sticking a return in the middle introduces an unexpected ordering dependency. -- Greg
On 25.04.2012 16:00, M.-A. Lemburg wrote:
So you don't have any classes that span hundreds of lines? I don't see how this is different in terms of editing difficulty. As for readability, at least you can see from the indentation that there's something special about the module code in question. You don't see that if there's a "return" scattered somewhere. cheers, Georg PS: I don't buy the "it's no problem with functions" argument. Even though I'm certainly guilty of writing 100+ line functions, I find them quite ungraspable (is that a word?) and usually try to limit functions and methods to a screenful.
On Wed, Apr 25, 2012 at 5:41 AM, M.-A. Lemburg <mal@egenix.com> wrote:
Nick Coghlan wrote:
On Wed, Apr 25, 2012 at 10:55 AM, Steven D'Aprano <steve@pearwood.info> wrote:
IMO, defining things twice in the same module is not a very Pythonic way of designing Python software.
Agreed, which is one problem with accelerator modules.
Left aside the resource leakage, it also makes if difficult to find the implementation that actually gets used,
Agreed, which is another problem with accelerator modules.
bypasses "explicit is better than implicit",
Agreed, which is is another problem with import * from accelerator So far, these reasons have been saying "Don't do that if you can help it", and if you do it anyhow, making the code stick out is better than nothing. (That said, making it stick out by having top-level definitions that are *not* at the far left is ... unfortunate.)
That is a real problem, but the answer is to be more explicit. try: from _foo import A except ImportError ... # define exactly A, as it will be needed or try: A except NameError: ... # define exactly A, as it will be needed or even create a wrapper that imports or defines your object the first time it is called...
Python is normally written with a top-to-bottom view in mind, where you don't expect things to suddenly change near the end.
Actually, I tend to (wrongly) view the module-level code as fixed declarations rather than commands, so that leaving a name undefined or defining it conditionally is just as bad. Having an alternative definition in the same location is less of a problem. I have learned to look for import * at the beginning and end of a module, but more special cases would not be helpful. -jJ
On Tue, Apr 24, 2012 at 8:55 PM, Steven D'Aprano <steve@pearwood.info> wrote:
- encourages or requires duplicate code and copy-and-paste programming
How does stopping a module definition early encourage duplicate code? Just because you might have to react differently after a full import vs a partial?
Today, if you successfully import a module, you know that all the top-level code in that module was executed.
No, you don't. On the other hand, the counterexamples (circular imports, some lazy imports, module-name clashes, top-level definitions dependent on what was already imported by other modules, etc) are already painful to debug. -jJ
On 04/24/2012 03:15 PM, Mark Shannon wrote:
Does it also allow returns in class bodies when you do that?
Weather or not it does. I think modules are closer to class bodies and return should be a syntax error in that case as well. They aren't functions and we shouldn't think of them that way. IMHO It would make Python harder to learn when the lines between them get blurred. Cheers, Ron
M.-A. Lemburg wrote:
Antoine Pitrou wrote:
Premature micro-optimizations. Just starting the interpreter runs imports, defines functions and classes, and binds resources which your script may never use. Lists and dicts are over-allocated, which you may never need. Ints and strings are boxed. Python is a language that takes Knuth seriously: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." In the majority of cases, a few more small inefficiencies, especially those that are one-off costs at startup, are not a big deal. In those cases where it is a big deal, you can place code inside if-blocks, factor it out into separate modules, or use delayed execution by putting it inside functions. It may not be quite so pretty, but it gets the job done, and it requires no new features.
You can also solve it by defining the slow version first, as a fall-back, then replace it with the fast version, if it exists: import os, sys from MyOtherPackage import foo, bar, baz class MyClass: ... def MyFunc(a,b,c): ... def main(): ... try: from MyModule_C_Extension import * except ImportError: pass if __name__ == '__main__': main() The advantage of this is that MyModule_C_Extension doesn't need to supply everything or nothing. It may only supply (say) MyClass, and MyFunc will naturally fall back on the pre-defined Python versions. This is the idiom used by at least the bisect and pickle modules, and I think it is the Pythonic way. -- Steven
On 24 April 2012 20:23, Eric Snow <ericsnowcurrently@gmail.com> wrote:
For what it's worth I've wanted this a couple of times. There are always workarounds of course (but not particularly pretty sometimes). Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On 24Apr2012 13:23, Eric Snow <ericsnowcurrently@gmail.com> wrote: | In a function you can use a return statement to break out of execution | in the middle of the function. With modules you have no recourse. | This is akin to return statements being allowed only at the end of a | function. [...] Something very similar came up some months ago I think. My personal suggestion was: raise StopImport to cleanly exit a module without causing an ImportError, just as exiting a generator raises Stopiteration in the caller. In this case the caller is the import machinery, which should consider this except a clean completion of the import. Cheers, -- Cameron Simpson <cs@zip.com.au> DoD#743 http://www.cskk.ezoshosting.com/cs/ Disclaimer: Opinions expressed here are CORRECT, mine, and not PSLs or NMSUs.. - Larry Cunningham <larry@psl.nmsu.edu>
Eric Snow, 24.04.2012 21:23:
Without having read through the thread, I think that code that needs this is just badly structured. All of the above cases can be fixed by moving the code into a separate (and appropriately named) module and importing conditionally from that. I'm generally -1 on anything that would allow non-error code at an arbitrary place further up in a module to prevent the non-indented module code I'm looking at from being executed. Whether the result of that execution makes it into the module API or not is a different question that is commonly answered either by "__all__" at the very top of a module or by the code at the very end of the module, not in between. Stefan
participants (22)
-
Antoine Pitrou -
Arnaud Delobelle -
Cameron Simpson -
Eric Snow -
Ethan Furman -
Georg Brandl -
Greg Ewing -
Jan Kaliszewski -
Jim Jewett -
M.-A. Lemburg -
Mark Shannon -
Matt Joiner -
Michael Foord -
Mike Meyer -
Nick Coghlan -
Rob Cliffe -
Ron Adam -
Ronald Oussoren -
Stefan Behnel -
Steven D'Aprano -
Terry Reedy -
Yuval Greenfield