Mailman 3 September 2011 - Python-ideas

return value of yield expressions
by H. Krishnan 16 Sep '11

16 Sep '11

Hi, I should start off by saying that my knowledge of PEP342 is less than a week old, and I cannot claim any use-case for what I am about to propose. The PEP itself says "In effect, a yield-expression is like an inverted function call; the argument to yield is in fact returned (yielded) from the currently executing function, and the "return value" of yield is the argument passed in via send()." One could, I guess, wrap a function to be 'called' via this mechanism: def genwrap(func): def gen(): ret = None while True: args, kwds = (yield ret) ret = func(*args, **kwds) del args, kwds g = gen() g.next() return g f = genwrap(func) f.send( (args, kwds) ) However, 'send' takes only one argument (and hence the poor syntax in the last statement). Has the option of the return value of the yield expression treated very much like function arguments been discussed? (a1, a2, a3 = default, *args, **kwds) = (yield ret) 'send' could support positional and keyword arguments. I guess this particular form would break backward compatibility but there might be other alternatives. Regards, Krishnan

9 32

Object interface to path names
by David Townshend 16 Sep '11

16 Sep '11

I've recently been developing a tool to track changes to a fairly large structured file system, and in the process got to thinking about how working with path names could be improved. The problems I've had with using just os and os.path have lead to three objectives of any new implementation: 1. Cleaner handling of paths names, specifically constructing path names without the need for a lot of nested os.path.join() and os.split() functions. 2. Allow a validation of paths names based on predefined rules. (Although this requirement might be very specific to my use case) 3. Allow caching of file attribute data so that queries do not have to wait the disk or network to respond (although at the cost of accuracy). The first can be met with behaviour as follows, basically handling paths as containers of sub-paths: >>> root = Path('/usr') >>> print([n for n in root]) ['bin', 'local', 'lib', 'share'] >>> print([n for n in root.dirs]) ['bin', 'local', 'lib', 'share'] >>> print([n for n in root.files]) [] >>> root['local/share'] Path('/usr/local/share') >>> root['/usr/local/share'] Path('/usr/local/share') >>> share = Path('/usr/local/share') >>> share in root True The second can be met by allowing Path to be subclassed and defining factory functions for the creation of sub-paths: >>> root = Path('/usr') >>> root.factory = my_factory_function >>> root['local/share'] MyPath('/usr/local/share') The third can be met be allowing all disk calls to be asynchonous: >>> path = Path('/home/david/newfile', async=True) >>> path.touch() >>> path.exists() False >>> time.sleep(2) >>> path.exists() True This could all be implemented in pure python using os and os.path, and threading for asynchonous calls. I haven't yet thought through a complete specification for Path, but I image it would need to contain functions such as exists(), isfile(), isdir(), stat(), walk(), and allow iterator access to children. Does anyone else see a usefulness for this? Regards David

6 10

threading : AttributeError: 'NoneType' object has no attribute 'start'
by Sagar Neve 14 Sep '11

14 Sep '11

I am new to the python and I need a small script to be written using threads which reads the specific files present in a directory and does some selections of lines and then write those lines into different files. However the error I am getting is: AttributeError: 'NoneType' object has no attribute 'start' Here is the code. Can somebody please help me. This is very urgent. def getFiles(fname): fname=preprocessLogFileDir + "/" + fname print fname fin = open(fname, "r") line=fin.readline() print line .... # some extraction logic. fout = open("out-single-file/1xx_2xx_total", "w") fout.write(line) fout.close() files=os.listdir(preprocessLogFileDir) cnt=1; for fname in files: print fname if fnmatch.fnmatch(fname,'*.log'): #and os.path.isfile(fname): thr = "thr" + str(cnt) print thr thr=getFiles(fname) thr.start() thr.join cnt = cnt + 1 else: print "\npreprocessLogFileDir/fname is either a directory or does not end with 'log'." print "Probabally not a pre-process file. Ignoring the file.\n\n"

4 4

List Revolution
by Christopher King 13 Sep '11

13 Sep '11

The first element in a list is element zero, the second is one, the third it two, and so on. This some times confuses newbies to the language or programming in general. This system was invited when single bits where precious. It's time to update. Keep in mind this is something for version 4, since its not reverse compatible. I say we make the first element 1, second 2, third 3, and so on. Other languages would follow. We are python, made for easiness and readability, and we are in the age where you can't even get something as small as a kilobyte USB. We must make first one, second 2, and third 3, like it is supposed to be. I give this: *+1*

24 38

Re: [Python-ideas] List Revolution
by Matt Billenstein 12 Sep '11

12 Sep '11

Ugh, I hate the idea of library code having to think about dealing with the possibility that you chose 42 as the starting index for some list you passed into a function in said library. Then I suppose we'll have to extend this to strings and tuples and so forth as well? Seriously, any further "solutions" for this non-problem are just mental masturbation -- please stop. m On Sun, Sep 11, 2011 at 11:05:28PM -0700, mike.w.meyer(a)gmail.com wrote: > Ok, how about enhancing the list type to allow specifying an offset, > defaulting to 0? The offset should be positive, and that many elements simple > aren't there. Asking for an element that isn't there, or a slice including > such an element, raises value error exception. The iterators & etc for this > list would all start at offset instead of zero. > -- > Sent from my Android tablet with K-9 Mail. Please excuse my brevity. > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > http://mail.python.org/mailman/listinfo/python-ideas -- Matt Billenstein matt(a)vazor.com http://www.vazor.com/

1 0

Re: [Python-ideas] List Revolution
by Massimo Di Pierro 12 Sep '11

12 Sep '11

Here is my proposal which hopefully would settle the issue to rest class List(list): """ >>> a=List() >>> for i in range(100000): a.append(i) >>> print a.first 0 >>> print a.second 1 >>> print a.third 2 >>> print a.twentieth 19 >>> print a.twentysecond 21 >>> print a.onehundredthirtyfifth 134 >>> print a.onethousandfivehundredandthirtyeighth 1537 """ def __getattr__(self,name): import re if name.endswith('first'): name = name[:-5]+'one' elif name.endswith('second'): name = name[:-6]+'two' elif name.endswith('third'): name = name[:-5]+'three' elif name.endswith('fth'): name = name[:-3]+'ve' elif name.endswith('hth'): name = name[:-3]+'th' elif name.endswith('ieth'): name = name[:-4]+'y' elif name.endswith('th'): name = name[:-2] subs = { 'one':'+1', 'two':'+2', 'three':'+3', 'four':'+4', 'five':'+5', 'six':'+6', 'seven':'+7', 'eigth':'+8', 'nine':'+9', 'ten':'+10', 'eleven':'+11', 'twelve':'+12', 'thirteen':'+13', 'fourteen':'+14', 'fiftheen':'+15', 'sixteen':'+16', 'seventeen':'+17', 'eighteen':'+18', 'nineteen':'+19', 'ten':'+10', 'twenty':'+20', 'thirty':'+30', 'fourty':'+40', 'fifthy':'+50', 'sixty':'+60', 'seventy':'+70', 'eighty':'+80', 'ninety':'+90', 'hundred':')*100+(', 'thousand':')*1000+(', 'million':')*1000000+(', 'billion':')*1000000000+(', 'trillion':')*100000000000+(', 'and',''} for key,value in subs.items(): name = name.replace(key,value) if '(' in name: name='('+name+')' name.replace('()','1') if not re.compile('[\d\+\*\(\)]+').match(name): return AttributeError try: return self[eval(name)-1] except: raise AttributeError I would like to stress that the use of eval in this case is safe as the expression is first validated against a regex. Massimo

6 11

Columns for pprint.PrettyPrinter
by Julian Berman 12 Sep '11

12 Sep '11

For the most part I find pprint to be lovely. Simple and does a useful thing that I don't want to have to write every time, especially if I'm using it just to debug. However, I often find that I'm using it to print out a long-length singly nested structure whose elements' reprs are not really that long, for which its output is cumbersome. Try to pprint.pprint(range(100)). Instead of seeing a long really skinny column, it'd be nice if it could quickly provide a way (combined with the width arg that it already takes) to split up the elements of the structure within the width, so that instead of seeing things like >>> pprint.pprint(range(30)) [0, 1, 2, ... ] it could be coerced into something like >>> pprint.pprint(range(30), columns=5) [0, 1, 2, 3, 4 3, 4, 5, 6, 7, ... ] or for something nested, which I'm less thrilled with, and haven't thought out how to implement unless you have a somewhat balanced structure, but for posterity: {"foo" : {"bar" : 1, {"hello" : 2, {"other" : 1, "baz" : 2, "world" : 1}, "thing" : 2, "foo" : 3}, "here" : 3}, ... } Obviously it's meant to be simple, the comment at the top of the module even says so, and doing something like ^ is easy enough, but for what it's good for (saving me from having to write code that makes my objects easier to debug by displaying them nicely), just making it do a bit more would make life easier.

3 2

relaxing keyword usage restrictions
by H Krishnan 11 Sep '11

11 Sep '11

Hi, Apologies if this has been asked already: why can't the usage of at least some keywords as identifiers be allowed with the context of the usage being used to identify if a keyword or a variable is being referred to. For example, if = 3 # if is identifier if if = 3: # first if is keyword, second if is identifer A more complex usage may require some rules: a = not or and and ==> parse left to right to interpret as a = ( (not or) and (and) ). 'or' and the last 'and' are identifiers While some of the usage may be confusing to the user, that is not for the language to dictate but for the user to choose. Almost all keywords have specific semantics which (in my limited understanding) do not conflict with their usage as identifiers. Regards, Krishnan

23 51

Re: [Python-ideas] List Revolution
by Guido van Rossum 11 Sep '11

11 Sep '11

On Sat, Sep 10, 2011 at 12:07 PM, Massimo Di Pierro <mdipierro(a)cs.depaul.edu> wrote: > Not to open a can of worms but why not use eval in this example? > It is faster. It is 5x more compact. It allows me to take advantage of the Python parser instead of reinventing the wheel. It is less error prone and easier to understand. It is a python keyword. It is unsafe (as the bug in your first version inadvertently illustrated :-). I don't actually quite follow what your example does (sorry), but, even though it is less powerful than exec(), eval() invokes the full Python parser+compiler, which can take quite a bit of time to start up and uses a fair bit of memory. Also in other Python versions this is often much slower than in CPython, or has a big startup overhead. But, of course, it is great for quick demos! -- --Guido van Rossum (python.org/~guido)

1 0

Re: [Python-ideas] List Revolution
by Massimo Di Pierro 11 Sep '11

11 Sep '11

On Sep 10, 2011, at 2:22 PM, Matt Billenstein wrote: > On Sat, Sep 10, 2011 at 02:14:13PM -0500, Massimo Di Pierro wrote: >> Not to open a can of worms but why not use eval in this example? >> It is faster. It is 5x more compact. It allows me to take advantage of the >> Python parser instead of reinventing the wheel. It is less error prone and >> easier to understand. It is a python keyword. > > I wouldn't worry too much about the implementation details of a clearly insane > idea... agreed > And eval generally has high "code-smell" -- it can lead to code injection exploits. Not in this example. So why not? Any code can introduce vulnerabilities if incorrect.

2 2