Mailman 3 March 2009 - Python-ideas

Re: [Python-ideas] [Python-Dev] Ext4 data loss
by Steven D'Aprano 12 Mar '09

12 Mar '09

On Thu, 12 Mar 2009 01:21:25 am Antoine Pitrou wrote: > Christian Heimes <lists <at> cheimes.de> writes: > > In my initial proposal one and a half hour earlier I suggested > > 'sync()' as the name of the method and 'synced' as the name of the > > flag that forces a fsync() call during the close operation. > > I think your "synced" flag is too vague. Some applications may need > the file to be synced on close(), but some others may need it to be > synced at regular intervals, or after each write(), etc. > > Calling the flag "sync_on_close" would be much more explicit. Also, > given the current API I think it should be an argument to open() > rather than a writable attribute. Perhaps we should have a module containing rich file tools, e.g. classes FileSyncOnWrite, FileSyncOnClose, functions for common file-related operations, etc. This will make it easy for conscientious programmers to do the right thing for their app without needing to re-invent the wheel all the time, but without handcuffing them into a single "one size fits all" solution. File operations are *hard*, because many error conditions are uncommon, and consequently many (possibly even the majority) of programmers never learn that something like this: f = open('myfile', 'w') f.write(data) f.close() (or the equivalent in whatever language they use) may cause data loss. Worse, we train users to accept that data loss as normal instead of reporting it as a bug -- possibly because it is unclear whether it is a bug in the application, the OS, the file system, or all three. (It's impossible to avoid *all* risk of data loss, of course -- what if the computer loses power in the middle of a write? But we can minimize that risk significantly.) Even when programmers try to do the right thing, it is hard to know what the right thing is: there are trade-offs to be made, and having made a trade-off, the programmer then has to re-invent what usually turns out to be a quite complicated wheel. To do the right thing in Python often means delving into the world of os.O_* constants and file descriptors, which is intimidating and unpythonic. They're great for those who want/need them, but perhaps we should expose a Python interface to the more common operations? To my mind, that means classes instead of magic constants. Would there be interest in a filetools module? Replies and discussion to python-ideas please. -- Steven D'Aprano

5 6

Adding a test discovery into Python
by Guilherme Polo 12 Mar '09

12 Mar '09

Hi, I believe it would be good to include a test discovery into Python, right now I notice all the following packages: bsdbb, ctypes, distutils, email, json, lib2to3 and lib-tk (tkinter in py3k) duplicate some form of test discovery that works for each one of them (there are also sqlite3 tests, but they are not using a real "test discovery"). In the future it is very likely that this "duplication count" increases, since from time to time new modules and packages get into Python. I can also feel the "idlelib" package starting getting tests, just making things worse. External packages would benefit from it too. Right now you either duplicate the test discovery once more (because your project is small enough that you don't want to use something specific for that), or you use nose, trial, py.test or whatever looks better for you. So.. is there any chance we can enter in agreement what features would be useful in a test discovery that could be included with Python ? I for myself do not have fancy wishes for this one, I would be happy with something that would collect unittests, inside packages and subpackages, with some fixed patterns and let me run them with test.test_support.run_unittests, or maybe something that would collect unittests and doctests and have something like run_tests in test.test_support. But then I believe this wouldn't be good enough to substitute any of the current tools, making the addition mostly useless. Thanks for reading, -- -- Guilherme H. Polo Goncalves

8 14

cd statement?
by Sturla Molden 12 Mar '09

12 Mar '09

When working with Python interactive shell (particularly IDLE started from Windows start menu), one thing I miss is a cd statement. Ok, I can do >>> import os >>> os.setcwd('e:\\work') But I keep feeling that Matlab's cd statement is more handy: >> cd e:\work One other feature that makes Matlab's shell more handy, is the whos statement. It lists all variables created form the shell, types, etc. Yes it is possible to get all local and global names in Python/IDLE, but that is not the same. The variables created interactively get hidden in the clutter. IDLE also lacks a command history. If I e.g. make a typo, why do I have to copy and paste, instead of just hitting the arrow button? Although cosmetically, these three small things keep annoying me. :-( Sturla Molden

9 9

Fwd: Adding a test discovery into Python
by Guilherme Polo 12 Mar '09

12 Mar '09

---------- Forwarded message ---------- From: Guilherme Polo <ggpolo(a)gmail.com> Date: Wed, Mar 11, 2009 at 8:04 PM Subject: Re: [Python-ideas] Adding a test discovery into Python To: Raymond Hettinger <python(a)rcn.com>, python-dev(a)python.org On Wed, Mar 11, 2009 at 7:37 PM, Raymond Hettinger <python(a)rcn.com> wrote: > [Christian Heimes] >>>> >>>> I'm +1 for a simple (!) test discovery system. I'm emphasizing on simple >>>> because there are enough frameworks for elaborate unit testing. > > Test discovery is not the interesting part of the problem. Interesting or not, it is a problem that is asking for a solution, this kind of code is being duplicated in several places for no good reason. > > Axiom: The more work involved in writing tests, the fewer > tests that will get written. At some point you will have to run them too, I don't think you want to reimplement the discovery part yet another time. -- -- Guilherme H. Polo Goncalves

3 3

float vs decimal
by William Dode 12 Mar '09

12 Mar '09

Hi, I just read the blog post of gvr : http://python-history.blogspot.com/2009/03/problem-with-integer-division.ht… And i wonder why >>> .33 0.33000000000000002 is still possible in a "very-high-level langage" like python3 ? Why .33 could not be a Decimal directly ? bye -- William Dodé - http://flibuste.net Informaticien Indépendant

2 4

Re: [Python-ideas] [Python-Dev] Formatting mini-language suggestion
by Raymond Hettinger 12 Mar '09

12 Mar '09

[Guido van Rossum] > I suggest moving this to python-ideas and > writing a proper PEP. Okay, it's moved. Will write up a PEP, do research on what other languages do and collect everyone's ideas on what to put in the shed. (hundreds and ten thousands grouping, various choices of decimal points, mayan number systems and whatnot). Will start with Nick's simple proposal as a starting point. [Nick Coghlan] > [[fill]align][sign][#][0][minimumwidth][,][.precision][type] Other suggestions and comments welcome. Raymond

3 2

Ruby-style Blocks in Python Idea
by tav 12 Mar '09

12 Mar '09

Hey all, I've come up with a way to do Ruby-style blocks in what I feel to be a Pythonic way: using employees.select do (employee): if employee.salary > developer.salary: fireEmployee(employee) else: extendContract(employee) I originally overloaded the ``with`` keyword, but on Guido's guidance and responses from many others, switched to the ``using`` keyword. I explain the idea in detail in this blog article: http://tav.espians.com/ruby-style-blocks-in-python.html It covers everything from why these are useful to a proposal of how the new ``do`` statement and __do__ function could work. Please note that the intention of this proposal is not to encourage iteration with blocks -- Python already does this rather elegantly -- but blocks are very useful for more than just iteration as various Ruby applications have shown. Let me know what you think. My apologies if I've missed something obvious. Thanks! -- love, tav plex:espians/tav | tav(a)espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian

17 43

Re: [Python-ideas] Ruby-style Blocks in Python Idea (alternative)
by Jan Kanis 11 Mar '09

11 Mar '09

This being python-ideas, I'll also have a go at it. Being someone who does like functional programing when used in limited quantities, I also think multi line lambdas (or blocks, whatever you call them) are a good thing if a good way could be found to embed them into Python. But I don't like the part of tavs proposal of handling them with a magic __do__ function. So what about this slightly modified syntax and semantics: def NAME(ARGS) [as OTHERNAME] with EXPRESSION_CONTAINING_NAME: BODY eg: def callback(param) as result with do_something(with_our(callback), other, args): print("called back with "+param) return foobar(param) this would be equivalent to def callback(param): print("called back with "+param) return foobar(param) result = do_something(with_our(callback), other_args) This example tav gave for url in urls: using pool.execute do: print "%s fetching %s" % (time.asctime(), url) httpc.get(url) print "%s fetched %s" % (time.asctime(), url) would be written as for url in urls: def fetch(url) with pool.execute(fetch): print "%s fetching %s" % (time.asctime(), url) httpc.get(url) print "%s fetched %s" % (time.asctime(), url) Compared to tavs proposal this would: - allow for use in expressions where the block is not the only argument, the block could even be passed in multiple parameter positions - make it clear that a function is being defined, and this syntax even allows for re-using the function later on, including testing etc. - not use any new keywords, as it is both syntactically and semantically an extension of the 'def' keyword. I think it also caters to those circumstances where you'd want to use a multiline lambda, without having the awkward reordering of first having to define the function and then passing it by name where the emphasis has to be on what you do with the function (analogous to decorators). It also does everything (I think) that Rubys blocks do. It does not solve the case where you'd want to pass multiple blocks/multiline lambdas to a function, but hey, you can't solve everything. There are several variations of this syntax that could also be considered, eg having the 'with' clause before the 'as' clause, or making the 'with' clause optional as well. Or doing 'using foo(callback, other, args) as answer with callback(param):', that would require a new keyword, but would allow the expression using the new function to be first, as it is presumably the most important part. my €0,02, - Jan

6 7

math module and complex numbers
by Sturla Molden 11 Mar '09

11 Mar '09

>>> import math >>> math.sqrt(-1) Traceback (most recent call last): File "<pyshell#7>", line 1, in <module> math.sqrt(-1) ValueError: math domain error I'd say math.sqrt(-1) should return 1j. Sturla Molden

4 4

Re: [Python-ideas] Ruby-style Blocks in Python Idea
by average 11 Mar '09

11 Mar '09

FW: Sorry, forgot to change the subject line for the sake of threaded mail readers and archives.... ---------- Forwarded message ---------- From: "Stephen J. Turnbull" <stephen(a)xemacs.org> Terry Reedy writes: > > Like many other Pythonistas I recognize that that an uninformative stock > > name of '<lambda>' is defective relative to an informative name that > > points back to readable code. What I dislike is the anonymity-cult > > claim that the defect is a virtue. > > That's unfair. Python has "anonymous blocks" all over the place, > since every control structure controls one or more of them. It simply > requires that they be forgotten at the next DEDENT. Surely you don't > advocate that each of them should get a name! > > I think this is a difference of cognition. Specifically, people who > don't want to name blocks as functions may not abstract processes to > signatures as easily, and reify whole processes (including all free > identifiers!) as objects more easily, as those who don't think naming > is a problem. > > > Since I routinely use standard names 'f' and 'g' (from math) to name > > functions whose name I do not care about, I am baffled (and annoyed) by... > > If the cognition hypothesis is correct, of course you're baffled. You > "just don't" think that way, while he really does. The annoyance can > probably be relieved by s/a developer/some developers/ here: I'm glad you're bringing out the cognitive aspect of this, because to me, though it may seem "gratuitously mystical or mystifying" , there is an essential epistemological component to this issue related to the [bidirectional] cognitive mapping of and between mathematical <-- and --> psychological identity and the confusion stems from the inability to frame the issue around a single linear "classical" construct as generally imposed by typed text (did you follow that?). Even writing this sentence confounds my multidimensional meaning which I'm trying to compress into standard language constructs. Normally, I'd use hand gestures and inflection to split out the two different orthogonal aspects of what I'm trying to convey which simply cannot occupy the same space at the same time, but (obviously) that luxury is unavailable here. So in the end, unless there's sufficient purchase in the listener, I have to be somewhat content sounding like a moron spouting gibberish. What Tav's proposal, in my mind, is aiming to do is provide greater syntactic support within Python so as to minimize cognitive gibberish when the code is reified in the mind of the viewer. Of course, it doesn't help that were culturally trained into VonNeuman architecture-thinking were such conflation of dimensionality is built into the hardware itself. Really, like Stephan is pointing out, "re-ification" *IS* the best analogy to help elucidate of this issue (better in German: Verdinglichung). See wikipedia's "Reification (Marxism)" (--though be prepared that, depending on your state of mind, it will either make sense or sound like its logic is [perfectly] backward, like some flipped bit because it borders that special interplay between subject-object.) These kind of [Anonymous] functions/code blocks explicitly tell the user that "This is NOT part of my program", yet (due to the classical, flat nature of standard computer programming) I must "include" (in a constrained way since I'm not able to include the context or externalized identity in which this code will be run) it here [in my editor window text] even though its logical geometry is orthogonal to my program. It's like a vortex out of flatland--an interface into a different dimension, hence it's difficulty in explaining it to the natives of flatlandia. To put a name on it puts an identity label upon something pointing in the wrong direction (i.e. to the surrounding code) which isn't *meant* to be an an independent block of usable code or be part of the social context of its surroundings. It's like seeing your own body's innards mapped inside-out into a computer program and calling it "marcos" while I continue to function normally in some other dimensionality in some mysterious way to magically maintain my normal cognition elsewhere. Better to see those innards as anonymous data (that for whatever reason I'm needing to interface to) even though they are perfectly functioning blocks with an identity elsewhere (i.e.: me). So, yes, "anonymity" can be a virtue from a given perspective. ...Seems to be a parallel to meta-programming but on the other side of the scale--instead of abstracting "upwards" into greater levels of abstraction, it abstracts sideways and downwards into levels of concreteness. Naming in both cases is problematic if you want to avoid the categorical errors easily made by the flatland of the typed text. gibberish? marcos

2 1