Mailman 3 April 2016 - Python-ideas

random.choice on non-sequence
by Rob Cliffe 19 Apr '16

19 Apr '16

It surprised me a bit the first time I realised that random.choice did not work on a set. (One expects "everything" to "just work" in Python! :-) ) There is a simple workaround (convert the set to a tuple or list before passing it) but ... On 2011-06-23, Sven Marnach posted 'A few suggestions for the random module' on Python-Ideas, one of which was to allow random.choice to work on an arbitrary iterable. Raymond Hettinger commented "It could be generalized to arbitrary iterables (Bentley provides an example of how to do this) but it is fragile (i.e. falls apart badly with weak random number generators) and doesn't correspond well with real use cases." Sven replied "Again, there definitely are real world use cases ..." I would like to revive this suggestion. I don't understand Raymond's comment. It seems to me that a reasonable implementation is better and more friendly than none, particularly for newbies. This is the source code for random.choice in Python 3.2 (as far as I know it hasn't changed since): def choice(self, seq): """Choose a random element from a non-empty sequence.""" try: i = self._randbelow(len(seq)) except ValueError: raise IndexError('Cannot choose from an empty sequence') return seq[i] And here is my proposed (tested) version: def choice(self, it): """Choose a random element from a non-empty iterable.""" try: it[0] except IndexError: raise IndexError('Cannot choose from an empty sequence') except TypeError: it = tuple(it) try: i = self._randbelow(len(it)) except ValueError: raise IndexError('Cannot choose from an empty iterable') return it[i] This works on (e.g.) a list/tuple/string, a set, a dictionary view or a generator. Obviously the generator has to be 'fresh' (i.e. any previously consumed values will be excluded from the choice) and 'throw-away' (random.choice will exhaust it). But it means you can write code like random.choice(x for x in xrange(10) if x%3) # this feels quite "Pythonic" to me! (I experimented with versions that, when passed an object that supported len() but not indexing, viz. a set, iterated through the object as far as necessary instead of converting it to a list. But I found that this was slower in practice as well as more complicated. There might be a memory issue with huge iterables, but we are no worse off using existing workarounds for those.) Downsides of proposed version: (1) Backward compatibility. Could mask an error if a "wrong" object, but one that happens to be an iterable, is passed to random.choice. There is also slightly different behaviour if a dictionary (D) is passed (an unusual thing to do): The current version will get a random integer in the range [0, len(D)), try to use that as a key of D, and raise KeyError if that fails. The proposed version behaves similarly except that it will always raise "KeyError: 0" if 0 is not a key of D. One could argue that this is an improvement, if it matters at all (error reproducibility). And code that deliberately exploits the existing behaviour, provided that it uses a dictionary whose keys are consecutive integers from 0 upwards, e.g. D = { 0 : 'zero', 1 : 'one', 2 : 'two', 3 : 'three', 4 : 'four' } d = random.choice(D) # equivalent, in this example, to "d = random.choice(list(D.values()))" if d == 'zero': ... will continue to work (although one might argue that it deserves not to). (2) Speed - the proposed version is almost certainly slightly slower when called with a sequence. For what it's worth, I tried to measure the difference on my several-years-old Windows PC, but is was hard to measure accurately, even over millions of iterations. All I can say is that it appeared to be a small fraction of a millisecond, possibly in the region of 50 nanoseconds, per call. Best wishes Rob Cliffe

20 37

Changing the meaning of bool.__invert__
by Antoine Pitrou 19 Apr '16

19 Apr '16

Hello, Booleans currently have reasonable overrides for the bitwise binary operators: >>> True | False True >>> True & False False >>> True ^ False True However, the same cannot be said of bitwise unary complement, which returns rather useless integer values: >>> ~False -1 >>> ~True -2 Numpy's boolean type does the more useful (and more expected) thing: >>> ~np.bool_(True) False How about changing the behaviour of bool.__invert__ to make it in line with the Numpy boolean? (i.e. bool.__invert__ == operator.not_) Regards Antoine.

25 55

Re: [Python-ideas] Have REPL print less by default
by Franklin? Lee 19 Apr '16

19 Apr '16

On Apr 19, 2016 5:48 AM, "Paul Moore" <p.f.moore(a)gmail.com> wrote: > > You say "should"? Do you mean that it is likely, or do you mean that it is > > what would happen in an ideal world? My college had CS students SSH into the > > department's Linux server to compile and run their code, and many teachers > > don't believe that students should start with fancy IDE featues like, er, > > syntax highlighting. > > I mean that that is what I hear people on lists like this saying as "a > reasonable beginner environment". It means that the people I've > introduced to Python (on Windows) have tended to end up using IDLE > (either from the start menu or via "edit with IDLE" from the right > click menu on a script). But are they perhaps tending to use IDLE because you were the one introducing them to it? Whether it is a recommended environment by the consensus of mailing lists is hardly indicative that IDLE is what people will start on. > My experience (in business, environments) is that people expect an IDE > when introduced to a new programming language, and IDLE, like it or > not, is what is available out of the box with Python. They expect it => they know what an IDE is. Programmers are already half-intermediate, especially programmers who use IDEs. > > You (and most regulars on this list) can adjust your shell to the way you > > like it, or use a more sophisticated shell, like IPython or bpython. On the > > other hand, changing shells and adding display hooks to site.py is not an > > option for those who don't know it's an option. > > As a "scripting expert" and consultant, I typically get asked to knock > up scripts on a variety of environments. I do not normally have what > I'd describe as "my shell", just a series of basic "out of the box" > prompts people expect me to work at. So no, the luxury of configuring > the default experience is *not* something I typically have. I suggested a command-line switch. > >> I view the REPL as more > >> important for intermediate or advanced users who want to quickly test > >> out an idea (at least, that's *my* usage of the REPL). > > > > But that doesn't answer my question: would the proposed change hurt your > > workflow? > > Yes. If I get a stack trace, I want it all. And if I print something > out, I want to see it by default. The REPL for me is an investigative > environment for seeing exactly what's going on. And why is it an insufficient option for you to type, for example, "_exfull()" to get the full trace? > (Also, having the REPL behave differently depending on what version of > Python I have would be a problem - backward compatibility applies here > as much as anywhere else). This is a concern, but not one that is enough, by itself, to justify the status quo. I disagree that it applies *anywhere near as much* here, since I don't see how it could break existing code. It will add a little adjustment and a little more work sometimes, but it doesn't require, say, refactoring a module, or every site page you ever downloaded.

2 2

pep 7 line break suggestion differs from pep 8
by Joseph Jevnik 18 Apr '16

18 Apr '16

I saw that there was recently a change to pep 8 to suggest adding a line break before a binary operator. Pep 7 suggests the opposite: > When you break a long expression at a binary operator, the operator goes at the end of the previous line, e.g.: > if (type->tp_dictoffset != 0 && base->tp_dictoffset == 0 && > type->tp_dictoffset == b_size && > (size_t)t_size == b_size + sizeof(PyObject *)) > return 0; /* "Forgive" adding a __dict__ only */ I imagine that some of the reasons for making the change in pep 8 for readability reasons will also translate to C; maybe pep 7 should also be updated.

3 2

Re: [Python-ideas] Add __main__ for uuid, random and urandom
by Wes Turner 18 Apr '16

18 Apr '16

On Apr 18, 2016 12:30 PM, "Wes Turner" <wes.turner(a)gmail.com> wrote: > > > > I think the users of oneline.py could be people that now write lots of > > bash scripts and work on the command line. So whenever someone asks a > > question somewhere about how to do X on the linux command line, we > > might have the answer: """ > > > > Q: On the linux commandline, how do I get only the filename from a > > full path that is in $FILEPATH > > > > A: Python has this. You can use the tools in os.path: > > > > Filename: > > $ oneline.py "os.path.basename('$FILEPATH')" > > > > Path to directory: > > $ oneline.py "os.path.dirname('$FILEPATH')" > > """ > > FILEPATH='for'"example');"'subprocess.call("cat /etc/passwd", shell=True)' sys.argv[1] (IFS=' ') stdin (~IFS=$'\n') ... * https://github.com/westurner/dotfiles/blob/develop/scripts/el * https://github.com/westurner/pyline/blob/master/pyline/pyline.py (considering adding an argument (in addition to the existing -m) for importlib.import_module)) > > > > > This might be more appealing than python -c. The whole point is to > > make Python's power available and visible for a larger audience. > > > > -Koos > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas(a)python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/

1 1

The next major Python version will be Python 8
by Victor Stinner 18 Apr '16

18 Apr '16

Hi, Python 3 becomes more and more popular and is close to a dangerous point where it can become popular that Python 2. The PSF decided that it's time to elaborate a new secret plan to ensure that Python users suffer again with a new major release breaking all their legacy code. The PSF is happy to announce that the new Python release will be Python 8! Why the version 8? It's just to be greater than Perl 6 and PHP 7, but it's also a mnemonic for PEP 8. By the way, each minor release will now multiply the version by 2. With Python 8 released in 2016 and one release every two years, we will beat Firefox 44 in 2022 (Python 64) and Windows 2003 in 2032 (Python 2048). A major release requires a major change to justify a version bump: the new killer feature is that it's no longer possible to import a module which does not respect the PEP 8. It ensures that all your code is pure. Example: $ python8 -c 'import keyword' Lib/keyword.py:16:1: E122 continuation line missing indentation or outdented Lib/keyword.py:16:1: E265 block comment should start with '# ' Lib/keyword.py:50:1: E122 continuation line missing indentation or outdented (...) ImportError: no pep8, no glory Good news: since *no* module of the current standard library of Python 3 respect the PEP 8, the standard library will be simplified to one unique module, which is new in Python 8: pep8. The standard library will move to the Python Cheeseshop (PyPI), to reply to an old and popular request. DON'T PANIC! You are still able to import your legacy code into Python 8, you just have to rename all your modules to add a "_noqa" suffix to the filename. For example, rename utils.py to utils_noqa.py. A side effect is that you have to update all imports. For example, replace "import django" with "import django_noqa". After a study of the PSF, it's a best option to split again the Python community and make sure that all users are angry. The plan is that in 10 years, at least 50% of the 77,000 packages on the Python cheeseshop will be updated to get the "_noqa" tag. After 2020, the PSF will start to sponsor trolls to harass users of the legacy Python 3 to force them to migrate to Python 8. Python 8 is a work-in-progress (it's still an alpha version), the standard library was not removed yet. Hopefully, trying to import any module of the standard library fails. Don't hesitate to propose more ideas to make Python 8 more incompatible with Python 3! Note: The change is already effective in the default branch of Python: https://hg.python.org/cpython/rev/9aedec2dbc01 Have fun, Victor

14 21

__getattr__ bouncer for modules
by Chris Angelico 17 Apr '16

17 Apr '16

Every now and then there's been talk of making it easier to subclass modules, and the most common use case that I can remember hearing about is descriptor protocol requiring code on the type. (For instance, you can't change a module-level constant into a property without moving away from the default ModuleType.) How bad would it be for the default ModuleType to have a __getattr__ function which defers to the instance? Something like this: def __getattr__(self, name): if '__getattr__' in self.__dict__: return self.__dict__['__getattr__'](name) raise AttributeError The biggest downside I'm seeing is that module attributes double as global names, which might mean this would get checked for every global name that ends up being resolved from the builtins (which is going to be a LOT). But I'm not sure if that's even true. Dumb idea? Already been thought of and rejected? ChrisA

4 6

Re: [Python-ideas] Adding a pipe function to functools
by Wes Turner 16 Apr '16

16 Apr '16

On Apr 16, 2016 12:07 AM, "Steven D'Aprano" <steve(a)pearwood.info> wrote: > > On Fri, Apr 15, 2016 at 01:51:45PM -0500, Wes Turner wrote: > > > * https://docs.python.org/3.1/howto/functional.html#the-functional-module > > > > from functional import compose, partial > > import functools > > multi_compose = partial(functools.reduce, compose) > > > > * https://docs.python.org/3.5/library/functools.html > > > > * functools.compose is gone! > > functools.compose never existed. The above example is > functional.compose, a third party library. my mistake. in my haste, I had thought that functional was functools. it looks like funcy has both curry() and compose(): * http://funcy.readthedocs.org/en/stable/funcs.html#curry #compose * https://github.com/Suor/funcy/blob/master/funcy/funcs.py #compose * https://github.com/Suor/funcy/blob/master/funcy/simple_funcs.py #curry and fn.py has __lshift__ and __rshift__ for F.__compose: * https://github.com/kachayev/fn.py/blob/master/fn/func.py * > > > > -- > Steve > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/

1 0

Adding a pipe function to functools
by Ed Minnix 16 Apr '16

16 Apr '16

Hello, I have been looking over the toolz library and one of the functions I like the most is pipe. Since most programmers are familiar with piping (via the Unix `|` symbol), and it can help make tighter code, I think it would be nice to add it to the standard library (such as functools). - Ed Minnix

5 6

Re: [Python-ideas] Welcome to the "Python-ideas" mailing list
by Richard Prosser 15 Apr '16

15 Apr '16

Is there any mileage in having a naming convention to indicate the type of a variable? I have never really liked the fact that the Python 'duck typing' policy is so lax, yet the new "Type Hints" package for Python 3 is rather clumsy, IMO. For example: github_response = requests.get('https://api.github.com/user', auth=('user', 'pass')) # Derived from http://docs.python-requests.org/en/master. The above request returns a Response <http://docs.python-requests.org/en/master/api/#requests.Response> object and so the variable has 'response' in its name. Likewise: word_count = total_words_in_file('text_file') where 'count' has been defined (in the IDE, by the user perhaps) as an Integer and the function is known to return an Integer, perhaps via a local 'count' or 'total' variable. I know that this has been attempted before but I think that an IDE like PyCharm could actually check variable usage and issue a warning if a conflict is detected. Also earlier usages of this 'Hungarian Notation' have largely been applied to compiled languages - rather strangely, in the case of known types - rather than an interpreted one like Python. Please note that I have shown suffixes above but prefixes could also be valid. I am not sure about relying on 'type strings' *within* a variable name however. Is this idea feasible, do you think? Thanks ... Richard PS Originally posted in https://intellij-support.jetbrains.com/hc/en-us/community/posts/207286145--…

5 4