Mailman 3 July 2010 - Python-ideas

Looking for a "batch" function
by Shane Hathaway July 18, 2010

July 18, 2010

Hi all, An operation I often want in my Python code is some kind of simple batch function. The batch function would take an iterator and return same-size batches from it, except the last batch, which could be smaller. Two parameters would be required: the iterator and the size of each batch. Here are some examples of what I would expect this batch function to do. Get batches from a list: >>> list(batch([1,2,3,4,5], 2)) [[1, 2], [3, 4], [5]] Get batches from a string: … [View More]

5 6

pop multiple elements of a list at once
by Diego Jacobi July 12, 2010

July 12, 2010

Hi. As recommended here: http://bugs.python.org/issue9218 I am posting this to this list. I am currently working with buffer in an USB device and pyusb. So when i read from the buffer of endpoint, i get an array.Array() list. I handle this chunk of data with a thread to send a receive the information that i need. In this thread, i load a list with all the information that is read from the USB device, and another layer with pop this information from the threads buffer. The thing i found is … [View More]

6 9

explicitation lines in python ?
by Daniel DELAY July 12, 2010

July 12, 2010

If we could explicitate a too complex expression in an indented next line, I would use this feature very often : htmltable = ''.join( '<tr>{}</tr>'.format(htmlline) for line in table) : # main line htmlline : ''.join( '<td>{}</td>'.format(cell) for cell in line) # explicitation(s) line(s) (Sorry if this has already been discussed earlier on this list, I have not read all the archives) ******* in details : List comprehension "<expression> for … [View More]x in mylist" often greatly improve readability of python programs, when <expression> is not too complex When <expression> is too complex (ex: nested lists), this become not readable, so we have to find another solution : a) defining a function expression(x), or an iterator function, wich will only used once in the code b) or droping this beautiful syntax to replace it the very basic list construction : newlist = [] for x in myiterable newlist.append(<expression>) I often choose b), but I dislike both solutions : - in solution a) function def can be far from list comprehension, in fact instructions to build the new list are split in two different places in the code. - solution b) seems a bit better to me, but the fact we build a new list from myiterable is not visible in a glance, unlike list comprehensions. Renouncing to list comprehension occurs rather often when I write python code I think we could greatly improve readability if we could keep list comprehension anywhere in all cases, but when necessary explicit a too complex expression in an indented line : htmltable = ''.join( '<tr>{}</tr>'.format(htmlline) for line in table) : # main line htmlline : ''.join( '<td>{}</td>'.format(cell) for cell in line) # explicitation(s) line(s) In the case the main line is the header of a "classical" indented block (starting with "for", "if", "with"...) , this idented block would simply follow the explicitation(s) line(s). The explicitations lines can be surely identified as the lines than begin with "identifier :" (when we are not in an unclosed dict) with open('data.txt') as f : if line in enumerate(mylist) : # main line mylist : f.read().strip().lower() # explicitation(s) line(s) print line # "classical" indented block Another possible use of "explicitations lines" is a coding style wich would start by "the whole picture" first, and completing with details after, wich is the usual way we mentally solve problems. Let's take an example : we want to write a function wich returns a multiplication table in a simle html document. When I solve this problem, il think a bit like that : - I need to return an html page. For that I need a "header" and a "body". My body will contain an "htmltable", wich be built from a "table" of numbers etc. My code could look like that : def tag(content, *tags): # little convenient function retval = content for t in tags: retval = '<{0}>{1}</{0}>'.format(t, retval) return retval def xhtml_mult_table(a, b): return tag(header + body, 'html') : header : tag('multiplication table', 'title') body : tag(htmltable, 'tbody', 'table', 'body') : htmltable : ''.join(tag(xhtmlline, 'tr') for line in table) : table : headerline + otherlines : headerline : [[''] + range(a)] otherlines : [[y] + [x*y for x in range(a)] for y in range(b)] xhtmlline : ''.join(tag(str(cell), 'td') for cell in line) This example is a "heavy use" of the "explicitation line" feature, to illustrate how it could work. I don't mean this should replace the "classical" syntax everywhere possible, but for me this would be for me a nice way to explicitate complex expressions from time to time, and the ability to use list comprehension everywhere I wan't. Daniel [View Less]

8 17

Re: [Python-ideas] explicitation lines in python ?
by Carl M. Johnson July 12, 2010

July 12, 2010

One more quick thought about the advantage of a where-clause. Often times, there is thought of creating the equivalent of Haskell-style pattern matching using decorators. For example PJE has worked on creating "generic functions." One problem with using a decorator for this is that for use cases more complicated than just matching on type, the matcher itself needs to be a function that looks at the arguments then returns true or false based on whether they match a pattern. So, a proper … [View More]

1 0

Using only patches for pulling changes in hg.python.org
by Tarek Ziadé July 7, 2010

July 7, 2010

Hello, If you follow python-checkins, you have probably notice and got annoyed this morning my +100 checkin mails in distutils2. I was lagging a bit on getting the GSOC students work pulled in, and, with the DVCS effect, you get 2/3 weeks of work in hg.python.org in a minute. :) Once CPython itself is in mercurial, we will probably have the same problem when people are pulling contributions. If you use a "hg pull" command it will get all commits from the third party, even if some if those … [View More]

6 9

Re: [Python-ideas] Using only patches for pulling changes in hg.python.org
by Tarek Ziadé July 6, 2010

July 6, 2010

On Sun, Jul 4, 2010 at 3:23 PM, Dirkjan Ochtman <dirkjan(a)ochtman.nl> wrote: ... > > Hmm, I don't think I agree on what you're saying. > > First, a changeset is a changeset is a changeset. If you exchange them as > patches or in some other way (by pulling or pushing or whatever) shouldn't > really matter. This is one of the things DVCS is good at, you can move csets > around different clones in many ways, and all clones are created equal. As far as I have … [View More]experienced, there's a back and forth game between the contributor and the commiter, leading to clone hell, unless the contributor uses mq, then apply it in his clone right before the contributor pulls the changes in. Using patches makes changes separated from the history for the time being, until they are merged. And are easy to read, review and understand. > Second, a noisy history is never good. So yes, pulling some kind of messy > history and pushing it to a central repo as-is is not a good idea. People > should polish their changesets so that each changeset can stand on its own. When you work on a feature, how do you polish a changeset without polluting your history or doing many clones ? (true question, I've been looking for that) > So yes, somewhere between it being a messy history of actual development and > it going into a central repo, it should be cleaned up. Ideally, the original > author should do that, but if he's not in a position to do so, the committer > should do it. Do you have an easy way to perform this cleanup ? Could you propose a process here ? I am bit skeptical that contributors will do this, whereas a patch policy makes sure we don't have to deal with this, and avoid asking people to have a high mercurial-fu. I am also skeptical that contributors are willing to digg into a clone to get what they want and/or check that it's fine to pull. It seems to me that patches are the universal format to propose a change and are easy to produce and consume. Contributors can use any process they want to create it, even without using mercurial. > > Third, if the result of cleaning up is a single cset, it should probably be > rebased before getting pushed to a central repo. If it's two or three csets, > rebase it. On the other hand, if it's 10 csets, actually doing an explicit > merge makes sense. The idea is not to clutter up the history with a merge > every other cset, but if the merge is hard/non-trivial it can make sense to > leave it explicit. > > Fourth, one-patch-per-issue is too restrictive. Small commits are useful > because they're way easier to review. Concatenate several small commits > leading up to a single issue fix into a single patch and it gets much harder > to read. Easy reviews are important, because a lot of valuable time is spent > reviewing. The simple example is a chain like refactor-refactor-fix (which > is IME quite common). Ideally each stage keeps the test suite passing and is > internally consistent, but moving towards a common goal (to fix the issue). > > So, I find your proposed policy somewhat vague and also not that attractive. > Cleaning up the history is certainly a good thing, but I don't think we have > to mandate a way for things to get into the repo. Mandating the use of > issues as a reference for each fix or enhancement could be useful, but seems > unnecessary. Yes, it's vague, I don't have a clear idea yet and I am not that experienced in hg latest features, so I am probably doing some steps wrong or ignore some shortcuts. But I have the strong feeling that without patches, we are heading for extra work for all parties unless we have a strong tutorial on how to contribute with hg.python.org, and that is proven to be very simple. side note: I am replying to the gname emails but I don't know if this works with the mailman thread as well.. Tarek [View Less]

11 14