Mailman 3 September 2013 - Python-ideas

Style for multi-line generator expressions
by Clay Sweetser 15 Sep '13

15 Sep '13

PEP 8 currently lacks any suggestions for how multi-line generator expressions and list comprehensions should be formatted. In the absence of any official style suggestion (that I can find), I suggest the style used the most in the standard library. [<expression> for <target_list> in <container> if <condition>] Note, lines could still be combined where it makes sense, eg, the first two lines could be combined if they aren't too long. -- "Evil begins when you begin to treat people as things." - Terry Pratchett

3 2

Should we improve `dir`?
by David Halter 15 Sep '13

15 Sep '13

I recently stumbled over `dir()` not working correctly in the case of classes: http://jedidjah.ch/code/2013/9/8/wrong_dir_function/ In short: `dir` doesn't list the `type` methods, which it should in my opinion, because there are very important attributes in there like `__name__` or `__bases__`. This led to some confusion in the past, e.g. http://www.gossamer-threads.com/lists/python/python/507363. The long version is in the above link. After discussions, I realized that I should probably bring this up in python-ideas, I think the current implementation can be very confusing for people trying to introspect classes with `dir`, which is IMHO its typical use case.

7 7

Re: [Python-ideas] Replace option set/get methods through the standard library with a ChainMap; add a context manager to ChainMap
by Oscar Benjamin 14 Sep '13

14 Sep '13

On 12 September 2013 11:06, Neil Girdhar <mistersheik(a)gmail.com> wrote: > > Exactly. I just assumed when I wrote my comment that this was a general > problem in the standard library, but as was pointed out to me, it seems to > be only a problem with numpy. My suggestion is for printoptions to be > implemented as a ChainMap to facilitate things on the numpy end of things as > well. Your suggestion doesn't seem unreasonable to me. However you're asking on the wrong mailing list: http://www.scipy.org/scipylib/mailing-lists.html Oscar

2 3

Re: [Python-ideas] AST Pretty Printer
by Ryan Gonzalez 14 Sep '13

14 Sep '13

Note: I didn't know who to reply to, so I just restarted the thread with the same subject. Here is the code: class astpp(ast.NodeVisitor): def __init__(self, tree): super(ast.NodeVisitor, self).__init__() self.indent = 0 self.visit(tree) def _print(self, text): print (' ' * self.indent + text) def generic_visit(self, node): self._print(node.__class__.__name__ + '(') self.indent += 1 for name, item in node.__dict__.iteritems(): if isinstance(item, ast.AST): self._print(name + '=') self.indent += 1 self.generic_visit(item) self.indent -= 1 elif isinstance(item, list): self._print(name + '=[') self.indent += 1 [self.generic_visit(attr) for attr in item] self.indent -= 1 self._print(']') else: self._print(name + '=' + str(item)) self.indent -= 1 self._print(')') Sample usage: astpp('''len('My friends are my power!')''') Output: Module( body=[ Expr( lineno=1 value= Call( col_offset=0 starargs=None args=[ Str( s=My friends are my power! lineno=1 col_offset=4 ) ] lineno=1 func= Name( ctx= Load( ) id=len col_offset=0 lineno=1 ) kwargs=None keywords=[ ] ) col_offset=0 ) ] ) -- Ryan

1 0

Re: [Python-ideas] multiprocessing and physical CPU cores count
by Giampaolo Rodola' 13 Sep '13

13 Sep '13

On Fri, Sep 13, 2013 at 10:14 AM, M.-A. Lemburg <mal(a)python.org> wrote: > On 12.09.2013 21:51, Giampaolo Rodola' wrote: > > On Thu, Sep 12, 2013 at 9:32 PM, Antoine Pitrou <solipsis(a)pitrou.net> > wrote: > > > >>> Then the question is whether having physical CPU cores count can be > >> useful. > >> > >> I suppose it doesn't hurt :-) I don't think it belongs specifically in > >> multiprocessing, though. Perhaps in the platform module? > >> > > > > I'd be +0.5 for multiprocessing because: > > > > - cpu_count() is already there > > - physical_cpu_count() will likely be used by multiprocessing users only > > > > ...but my main concern was first figuring out whether it might actually > > make sense to distinguish between virtual and physical CPUs in a real > world > > app. > > I'm with Antoine here: both APIs would make more sense in the > platform module. In the end it appears the os module would probably be better as cpu_count() already ended up there (http://bugs.python.org/issue17914) as pointed out by Victor a couple of emails ago. I have the impression no one is opposed so I can probably start working on a patch and submit it on the bug tracker. > Do you need C code for determining the physical count ? Yes, except on Linux where you'll just read /proc/cpuinfo. > >> (unless you want to contribute psutil to the stdlib?) > > > > > > That's something I'd be happy to do if there's general approval but I > guess > > that's for another thread. > > I'd love to see psutils in the stdlib, but also be warned: once > the code lives in the stdlib, > > a) making changes is difficult and adding new features as well, > > b) you are bound by the Python release cycle. > > For a package such psutil, it may actually be better to keep it > outside the stdlib, since the outside world changes regularly > and doesn't adhere to the Python release cycle or feature > for patch level releases ;-) > Yeah, you're probably right,and there's at least a couple of high priority functionalities I'd like to add first (to say one: dragonfly/open/net BSD support). --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/

1 0

multiprocessing and physical CPU cores count
by Giampaolo Rodola' 13 Sep '13

13 Sep '13

This is a follow up of a feature request which recently appeared on psutil bug tracker: https://code.google.com/p/psutil/issues/detail?id=427 I don't know whether the proposal makes sense for psutil per-se but it certainly made me think about multiprocessing.cpu_count() and the fact that it currently returns the number of virtual CPUs (physical + logical). Given that multiple processes cannot take any advantage of hyper threading technology then maybe it makes sense for multiprocessing to expose a physical_cpu_count() function in order to preemptively figure out how many processes to spawn. Same thing is discussed here: https://groups.google.com/forum/#!msg/nzpug/_5sFW9BEMQ4/Y4laXRNlXkMJ Thoughts? --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/

9 13

Replace option set/get methods through the standard library with a ChainMap; add a context manager to ChainMap
by Neil Girdhar 12 Sep '13

12 Sep '13

With numpy print options, for example, the usual pattern is to save some of the print options, set some of them, and then restore the old options. Why not expose the options as a ChainMap called numpy.printoptions? ChainMap could then expose a context manager that pushes a new dictionary on entry and pops it on exit via, say, child_context that accepts a dictionary. Now, instead of: saved_precision = np.get_printoptions()['precision'] np.set_printoptions(precision=23) do_something() np.set_printoptions(precision=saved_precision) You can do the same with a context manager, which I think is stylistically better (as it's impossible to forget to reset the option, and no explicit temporary invades the local variables): with np.printoptions.child_context({'precision', 23}): do_something() Best, Neil

2 1

Idea: Compressing the stack on the fly
by Ram Rachum 12 Sep '13

12 Sep '13

Hi everybody, Here's an idea I had a while ago. Now, I'm an ignoramus when it comes to how programming languages are implemented, so this idea will most likely be either (a) completely impossible or (b) trivial knowledge. I was thinking about the implementation of the factorial in Python. I was comparing in my mind 2 different solutions: The recursive one, and the one that uses a loop. Here are example implementations for them: def factorial_recursive(n): if n == 1: return 1 return n * factorial_recursive(n - 1) def factorial_loop(n): result = 1 for i in range(1, n + 1): result *= i return result I know that the recursive one is problematic, because it's putting a lot of items on the stack. In fact it's using the stack as if it was a loop variable. The stack wasn't meant to be used like that. Then the question came to me, why? Maybe the stack could be built to handle this kind of (ab)use? I read about tail-call optimization on Wikipedia. If I understand correctly, the gist of it is that the interpreter tries to recognize, on a frame-by-frame basis, which frames could be completely eliminated, and then it eliminates those. Then I read Guido's blog post explaining why he doesn't want it in Python. In that post he outlined 4 different reasons why TCO shouldn't be implemented in Python. But then I thought, maybe you could do something smarter than eliminating individual stack frames. Maybe we could create something that is to the current implementation of the stack what `xrange` is to the old-style `range`. A smart object that allows access to any of a long list of items in it, without actually having to store those items. This would solve the first argument that Guido raises in his post, which I found to be the most substantial one. What I'm saying is: Imagine the stack of the interpreter when it runs the factorial example above for n=1000. It has around 1000 items in it and it's just about to explode. But then, if you'd look at the contents of that stack, you'd see it's embarrassingly regular, a compression algorithm's wet dream. It's just the same code location over and over again, with a different value for `n`. So what I'm suggesting is an algorithm to compress that stack on the fly. An algorithm that would detect regularities in the stack and instead of saving each individual frame, save just the pattern. Then, there wouldn't be any problem with showing informative stack trace: Despite not storing every individual frame, each individual frame could still be *accessed*, similarly to how `xrange` allow access to each individual member without having to store each of them. Then, the stack could store a lot more items, and tasks that currently require recursion (like pickling using the standard library) will be able to handle much deeper recursions. What do you think? Ram.

13 20

Re: [Python-ideas] Replace numpy get_printoptions/set_printoptions, and similar patterns with a ChainMap; add a context manager to ChainMap
by Neil Girdhar 11 Sep '13

11 Sep '13

Just to be clear, my proposal is to replace all such get/set options patterns throughout Python's standard library. On Wednesday, September 11, 2013 5:13:55 PM UTC-4, Neil Girdhar wrote: > > With numpy, the usual pattern is to get_printoptions, set some of them, > and then restore the old options. Why not expose the options in a ChainMap > as numpy.printoptions? ChainMap could then expose a context manager that > pushes a new dictionary on entry and pops it on exit via, say, > child_context that accepts a dictionary. Now, instead of: > > saved_precision = np.get_printoptions()['precision'] > np.set_printoptions(precision=23) > do_something() > np.set_printoptions(precision=saved_precision) > > You can do the same with a context manager, which I think is stylistically > better (as it's impossible to forget to reset the option, and no explicit > temporary invades the local variables): > > with np.printoptions.child_context({'precision', 23}): > do_something() > > Best, > > Neil >

1 0

Python Sound API (Was: Cross Platform Python Sound Module/Library)
by anatoly techtonik 11 Sep '13

11 Sep '13

On Sat, Apr 27, 2013 at 11:59 PM, M.-A. Lemburg <mal(a)egenix.com> wrote: > On 27.04.2013 22:19, anatoly techtonik wrote: >> On Sat, Apr 27, 2013 at 1:51 PM, Antoine Pitrou <solipsis(a)pitrou.net> wrote: >> >>> On Fri, 26 Apr 2013 20:39:30 -0700 >>> Andrew Barnert <abarnert(a)yahoo.com> wrote: >>>> On Apr 26, 2013, at 16:59, Greg Ewing <greg.ewing(a)canterbury.ac.nz> >>> wrote: >>>> >>>>> Oleg Broytman wrote: >>>>>> Are there cross-platform audio libraries that Python could wrap? >>>>> >>>>> There's OpenAL: >>>>> >>>>> http://connect.creativelabs.com/openal/default.aspx >>>>> >>>> There's actually a bunch of options. >>>> >>>> The hard question is picking one and endorsing it as "right", or at least >>>> "good enough to enshrine in stdlib ala tkinter". >>> >>> When you notice how "good enough" tkinter is (and has been for 10 >>> years at least), you realize the trap hidden in this question. >>> >>> Really, see my message earlier in this thread. This is better left to >>> third-party libraries (which already exist, please do some research). >>> >> >>>From the other side if 80% of cases can be covered without Python packaging >> problems - that's already an advantage. For example most people find date / >> time functionality in Python enough to avoid using mxDateTime as a >> dependency. As for audio, most people find it insufficient. > > I'm not sure whether 3D audio support is really needed as core > feature in a general purpose programming language ;-) > > I'd suggest to have a look at http://www.libsdl.org/, which can > be used from Python via http://pygame.org/ 3D audio support is not the basic common cross-platform base layer. Many devices that run Python are mono at all. What stdlib should concentrate on are basic audio operations needed by people with accent on pure Python implementation of everything that not represents system layer. Safe sound synthesis and sound output in canonical format. If people need advanced algorithms and operations, they are free to use SDL2, OpenAL, FFmpeg and other libs that are inherently insecure due to amount of low level C code. Audio even on Android devices doesn't require any advanced privileges. It is a basic need for many programs and attraction for many creative people who may use Python as an auxiliarry language in their works. >From stdlib I also expected abstract scheduling and buffering algorithms with explanations and documentation with pictures. I expect super simple API for all basic cases. Ability to use this API on any platform. I'd expect Audio API to be multi-level. Level 1: Beep - make audio signal to attract attention with the most basic level OS provides Level 2: Customize audio signal to attract attention (query signals, chose, beep) Level 3: Play pre-rendered waveform (such as WAV file) with the most basic OS level (default format) Level 4: Play continuous pre-rendered stream Level 5: Mix pre-renderered streams and waveforms Level 6: Synthesize sound in pure Python Level 7: Synthesize sound indirectly (using GPU, MIDI interfaces, external libs, ...) Level 8: Audio device control - formats, channels, volumes - everything hardware specific -- anatoly t.

1 0