Mailman 3 December 2013 - Python-ideas

Easily reference a single unittest from cmdline
by Giampaolo Rodola' 09 Dec '13

09 Dec '13

During the development process it is very common to implement a new functionality, write a test for it and then run tests. I don't want to run *all* tests though, but only the new one which I just wrote. Currently unittest module lets you do this via cmdline with: python -m unittest test.test_module.TestClass.test_method This is not very practical though as you have to figure out the name of the module (easy) and the name of the test class (something which I *never* remember). Proposal is to allow a wildcard notation like this: python -m unittest test.test_module.*test_method* python -m unittest *test_method* What I expect from unittest is to execute only the test methods matching "*test_method*". This is related to an old proposal I raised 3 years ago: https://mail.python.org/pipermail/python-ideas/2010-August/007992.html After that dicussion I ended up coming up with a decorator for skipping tests: http://code.activestate.com/recipes/578234-unittestskip_others-decorator/?i… ...but I never find it really practical and I ended up abandoning it. In retrospective, I think the cmdline is the right place from where such a thing should be controlled. Thoughts? --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/

4 4

Automatically turning __prepare__ into a classmethod.
by Antony Lee 09 Dec '13

09 Dec '13

Just a small idea... __new__ is automagically turned into a staticmethod, because other interpretations don't make sense. Likewise, perhaps __prepare__ could be automagically turned into a classmethod? Antony

1 0

why is not request in httplib pretty printed?
by Matěj Cepl 07 Dec '13

07 Dec '13

Hi, looking at http://hg.python.org/cpython/file/543c76769c14/Lib/http/client.py#l847 (but this code has been same since like forever) I see that the HTTP request is NOT pretty printed: if self.debuglevel > 0: print("send:", repr(data)) whereas response in effect (because every header is printed separately) is. Wouldn't it be better to pretty print the request as well? Otherwise I get quite unreadable debugging logs like the following (notice how much response is more readable than request). Matěj matej@wycliff: urllib2_kerberos (next *%)$ python test_trac.py send: 'GET /desktopqe-backlog/login HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: someserver.redhat.com\r\nConnection: close\r\nUser-Agent: Python-urllib/2.7\r\n\r\n' reply: 'HTTP/1.1 401 Authorization Required\r\n' header: Date: Fri, 06 Dec 2013 17:11:13 GMT header: Server: Apache header: WWW-Authenticate: Negotiate header: WWW-Authenticate: Basic realm="Kerberos Login" header: Content-Length: 483 header: Connection: close header: Content-Type: text/html; charset=iso-8859-1 2013-12-06 18:11:15,406 DEBUG inside http_error_401 2013-12-06 18:11:15,407 DEBUG retry count: 1 2013-12-06 18:11:15,407 DEBUG req.get_host() returned someserver.redhat.com 2013-12-06 18:11:15,407 DEBUG authGSSClientInit() succeeded 2013-12-06 18:11:15,486 DEBUG authGSSClientStep() succeeded 2013-12-06 18:11:15,486 DEBUG authGSSClientResponse() succeeded send: 'GET /desktopqe-backlog/login HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: someserver.redhat.com\r\nConnection: close\r\nAuthorization: Negotiate YIICdQYJKoZIhvcSA---some-junk--GZDggzt\r\nUser-Agent: Python-urllib/2.7\r\n\r\n' reply: 'HTTP/1.1 302 Found\r\n' header: Date: Fri, 06 Dec 2013 17:11:14 GMT header: Server: Apache header: WWW-Authenticate: Negotiate YIGZBgkqhkiG9xIB--other-junk--NOxMdOR/5 header: Location: https://someserver.redhat.com/desktopqe-backlog header: Content-Length: 0 header: Pragma: no-cache header: Cache-Control: no-cache header: Expires: Fri, 01 Jan 1999 00:00:00 GMT header: Set-Cookie: trac_auth=148d029eb321e9a0782fd30ca3f9d17b; Path=/desktopqe-backlog; secure header: Connection: close header: Content-Type: text/plain; charset=UTF-8 send: 'GET /desktopqe-backlog HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: someserver.redhat.com\r\nConnection: close\r\nUser-Agent: Python-urllib/2.7\r\n\r\n' reply: 'HTTP/1.1 200 OK\r\n' header: Date: Fri, 06 Dec 2013 17:11:15 GMT header: Server: Apache header: Cache-Control: must-revalidate header: Expires: Fri, 01 Jan 1999 00:00:00 GMT header: Content-Length: 9591 header: Set-Cookie: trac_form_token=a6eda80ba3f4ead82093a0af; Path=/desktopqe-backlog; secure header: Set-Cookie: trac_session=213441ab7934999739979a04; expires=Thu, 06-Mar-2014 17:11:15 GMT; Path=/desktopqe-backlog; secure header: Connection: close header: Content-Type: text/html;charset=utf-8 2013-12-06 18:11:17,190 DEBUG www-authenticate header not found 2013-12-06 18:11:17,190 CRITICAL mutual auth failed. No negotiate header matej@wycliff: urllib2_kerberos (next *%)$

2 2

Attribute docstrings [was: One-line abstractmethod function?]
by Stephen J. Turnbull 07 Dec '13

07 Dec '13

Reply-To set to python-ideas(a)python.org. Terry Reedy writes: > For data attributes, which are usually mutable, it should be attached to > the attribute *concept*, which is represented by the name, rather than > the current but usually changeable value. Values are usually already > documented either by a value representation or a docstring. This could > be done with a string subclass that is used as needed. > > For methods, the value is nearly always constant. When multiple methods > share the same function, they usually also share the same name, and > represent the same concept. Aside: Properties are which? Data, or method? It's also not clear to me that "def first (self): return self.values[0]" is unlikely to be used for completely different purposes than getting the head of a list. I conclude the considerations above are mostly red herrings. The important thing, I suppose, is that the names of attributes defined in a class are not mutable. This means that their docstrings can be kept in a completely separate dict (or other string -> string mapping), which could even be stored in a separate file. (Emacs Lisp uses this to good effect. The DOC file for XEmacs is 1.6MB; for GNU Emacs it's 2.4MB.) Of course it has its problems, but they're pretty minor.

1 0

Replacing the if __name__ == "__main__" idiom (was Re: making a module callable)
by Eric Snow 06 Dec '13

06 Dec '13

On Fri, Nov 22, 2013 at 1:44 PM, Gregory P. Smith <greg(a)krypto.org> wrote: > It'd be nice to formalize a way to get rid of the __name__ == '__main__' > idiom as well in the long long run. Sure everyone's editor types that for > them now but it's still a wart. Anyways, digressing... ;) This has come up before and is the subject of several PEPS. [1][2] The current idiom doesn't bother me too much as I try not to have files that are both scripts and modules. However, Python doesn't make the distinction all that clear nor does it do much to encourage people to keep the two separate. I'd prefer improvements in both those instead, but haven't had the time for any concrete proposal. FWIW, aside from the idiom there are other complications that arise from a module that also gets loaded in __main__ (run as a script). See PEP 395 [3]. -erc [1] http://www.python.org/dev/peps/pep-0299/ [2] http://www.python.org/dev/peps/pep-3122/ [3] http://www.python.org/dev/peps/pep-0395/ (sort of related)

34 90

check interfaces, isducktype(X, A)
by Gregory Salvan 05 Dec '13

05 Dec '13

Hi, I've made a try of a function that check object/class members for duck typing. For now I've basically called it isducktype(X, A) it returns true if: - X has all attributes of A - X methods have same number of arguments than corresponding A methods or if A.__ducktypecheck__(X) returns True Behaviour looks like isinstance and issubclass (support tuples...). To test it: The patch with example is here: https://gist.github.com/apieum/7751279 Or you can use my branch here: https://bitbucket.org/gsalvan/cpython/commits/branch/default Any opinion about a feature like this ?

10 22

2 more admins for the list
by Brett Cannon 04 Dec '13

04 Dec '13

FYI both Ethan Furman and Tal Einat have joined Titus and myself in administering this list. And while I'm thinking about it, just a reminder about how to let us know about CoC violations. Please email/cc python-ideas-owner@ when you think there has been an infraction. There is enough volume on the list you can't assume we read every email. We also try to stay objective on this topic so chances are we won't bring up any CoC issues in order to stay neutral.

1 0

start, test, init
by spir 02 Dec '13

02 Dec '13

Hello, This is a proposal and opinion I have thought at sending for a long time already, but did not because it is not exactly proper to Python (there are a few more of this category ;-). What decided me is the thread about the idiom "if __name__ == '__main__':", which had much success! Below a short summary or my views, followed by a longer series of comments. Summary: There should be 3 top-level functions predefined by the language: * 'start' : start func for a module run as an app (or 'main') * 'test' : main func of a module's test suite * 'init' : module init code, rather when imported In my view, all 3 correspond to clearly definite & distinct functionalities. Each of them provides clarity and all together permit getting rid of "lost code" roaming around at the top-level of modules; which I dislike, even for scripts; however, the proposal does not force anyone to follow such a style. Every module using such functions is then a set of definitions: assignments, def's, classes, plus imports and such. Possibly 1, 2, 3 of them are language-defined main functions. Very clean... The execution logics would be: * if imported module: ~ run init if any ~ else, run nothing * if executed module: ~ run test, if any (see below) ~ else, run start, if any ~ else, run nothing (error?) What do you think? === why start? === It looks nice to have hello world reduced to: print("Hello, world!") However, this is a very superficial nicety. I prefere the (very big!) complication of: def start(): print("Hello, world!") This code is self-understanding even for a novice programmer and nicely introduces some structural feature of the language. [0] This function 'start' would have an optional argument for command-line args; it may indeed also return an exit code. About the name 'start', well, it's a question of meaning; and to avoid confusion I'd rather reserved 'main' for a package's main module. Also, every of these 3 funcs is a 'main' one, depending on the actual case. But I would *not* fight on this point, call it 'main' if you like. (In assembly, it's commonly start as well: both ends of the expressivity scale meet here ;-). When compiling with start, in standard the language may also automagically strip out development or control instructions, like assertions. (There also may be a differenciated debug print command... but that's another point. Also different error messages for end-users.) === why test? === If you practice testing (by code), you know why. Nearly all of my modules end up ending with: # === t e s t ======================= def test_abc (): ... def test_def (): ... def test_ghi (): ... ... def test (): test_abc() test_def() test_ghi() ... if __name__ == '__main__': test() Then, I comment out the last 2 lines when all runs fine; or else, if it's the main module of an app, I replace it with a call to start. This is a second form or running a module as a stand-alone prog (from the command-line). Actually, I guess, anyone practicing a minimum of testing may so-to-say press the 'test' button constantly, far more often than we lauch an app in usage or trial mode (if ever it's an app). We switch to normal usage execution once only per development phase, when all is fine and we prepare a user release. This is why, in standard, when both exist, 'test' has precedence over 'start'. There may be a builtin config var (eg __main__) to set (eg __main__ = start), or any other to switch to start (but preferably from code, see also [1]). This function 'test' _may_ have an optional argument for command-line args; it may also return an exit code, here meaning test success / failure (why not number of failures?). Args may be handy to drive testing differently: exact funcs to run, form of output, depth of testing..., in a purely user-defined way (no language-defined meaning, else we'd never stop arguing on the topic; a typical bikeshed issue). [1] === why init? === There is much less use, in my personal practice, for such an init function typically run when a module is imported. (I know it from the D language.) But I guess whenever we need it, having it is very nice. I simulate it (1) to init program elements from external data (2) to import, scan, process, big data files like unicode tables, avoiding huge code-data files and/or security issues (3) for a usage similar to compile-time computations in static langs that provide that. Anyway, it is a clearly defined functionality, just like start & test. === flexibility === An init func may be run by test or start, conditionnally or not. A test func may be run by init or start; maybe self-testing just once on first launch of a just-installed app or package. === alternative === Apart from clarity and practicality (see below), such flexibility is also why I prefere such predefined function names to a proposal by Steven D'Aprano, on the mentionned thread, of builtin conditionals like: if __main__: ... if is_main(): ... if is_main: ... We could however trivially extend this proposal with eg is_tested & is_imported builtin conditionals. Why not chose the direct simplicity of builtin func names, though? This is a common Python usage (it would also introduce novices to this idea of special func names, then they are ready for "magic methods"). Another issue is practicle: writing or not such a function, or chosing between test and start, is something we control from code itself. To launch tests for instance, there is thus no need to run a module with a special command-line option, or to modify it in our favorite programming editor's settings, or to change environment variables or python config files... all pretty annoying things to do (and I never remember which is to be done, where, how exactly...). Instead, predefined conditionnals depend on such settings or data external to the code. [2] === why simple names? === Again, I would not fight for simple names; it may well in fact be preferable to have weird names with underscores. In my view, the need in python for names like __all__ or __str__ is due to the fact that there is no difference in the language between defining and redefining a symbol. If Python used eg ':=' to redefine symbols, then: foo = ... # error, symbol 'foo' already exists foo := ... # ok, intentional redefinition of builtin 'foo' Waiting for this, it may indeed be preferable to have weird names. === trial mode === There is one mode not covered by this: what I call trial, as opposed to in-code tests. This corresponds more or less to a beta-version: a simulation of normal usage by developpers or (power-)users, to further find bugs or more generally control the software as is. I don't know of cases where this would require a special dedicated main func: this is rather opposite to the idea of checking the program really as is. But there may be code parts interpreted differently or conditionnally compiled, differently from the case of an actual user release: assertions, debug prints, stats, benchmarking, profiling... all kinds of "meta-programming" issues. === all together === ''' d o c ''' import foo __all__ = ... code... code... code... def test_abc (): ... def test_def (): ... def test_ghi (): ... ... def test (): test_abc() test_def() test_ghi() ... def init (): ... # __main__ = start def start (args): init() # process args ... What do you think? (bis) Denis [0] Python may lack such higher-level code structure, more generally; this is an open question. [1] This feature is for command-line args fans, but I prefere to control testing from code itself. For information, I have 2 std test config params, which I define in code; whether to: * write successful checks also, for comparison * continue on check failures, or else stop (They affect a custom check func, similar to an assert, used for testing only.) Turning them to off transforms a module's diagnosis test suite into (part of) a regression test suite. [2] More generally, I want to control all what concerns code and its development from code itself. Command-line args and such (better command-line args only) are good only for very special needs like producing doc, a parse-tree, compile-only, profiling, etc. I definitely dislike languages that force people to constantly change the command line to compile/run or worse change env vars or edit config files (and I don't even evoque makefiles and such...).

10 22

string codes & substring equality
by spir 02 Dec '13

02 Dec '13

Hello, Coming back to python after a long time. My present project is about (yet another) top-down matching / parsing lib. There are 2 issues that, I guess, may be rather easily solved by simple string methods. The core point is that any scanning / parsing process ends up, a the lowest level, constantly comparing either single-char (rather single-code) substrings or constant (literal) substrings of the source string. This is the only operation which, when successful, actually advances in the source. Thus, it is certainly worth having it efficient, or at the minimum not having it needlessly inefficient. I suppose the same functionalities can be highly useful in various other use cases of text processing. Note again that I'm rediscovering Python (with some pleasure :-), thus may miss known solutions -- but I asked on the tutor mailing list. In both cases, I guess ordinary idiomatic Python code actually _creates_ a new string object, as a substring of length 1 or more, which is otherwise useless; for instance: if s[i] == char: # match ok -- object s[i] unneeded if s[i:j] == substr: # match ok -- object s[i:j] unneeded What is actually needed is just to check for equality (or another check about a code, see below). The case of single-code checking appears when (1) a substring happens to hold a single code (meaning it represents a simple or precomposed unicode char) (2) when matching a char from a given set, range, or more complex class (eg in regex [a-zA-Z0-9_-']). In all cases, what we want is tocheck the code: compare it to a constant value, check whether it belongs to a set of value, or lies inside a given range. We need the code --not a single-code string. Ideally, I'd like expressions like: c = s.code(i) # or s.ord(i) or s.ucode(i) [3] # and then one of: if c = code: # match ok if c in codes: # match ok if c >= code1 and c <= code2: # match ok The builtin function ord(char) does not do the job, since it only works for a single-char string. We would again need to create a new string, with ord(s[i]). The right solution apparently is a string method like code(self, i) giving the code at an arbitrary index. I guess this is trivial. I'm surprised it does not exist; maybe some may think this is a symptom there is no strong need for it; instead, I guess people routinely use a typical Python idiom without even noticing it creates a unneeded string object. [2] [3] What do you think? A second need is checking substring equality against constant substrings of arbitrary sizes. This is similar to startswith & endswith, except at any code index in the source string; a generalisation. In C implementation, it would probably delegate to memcomp, with a start pointer set to p_source+i. On the Python side, it may be a string method like sub_equals(self, substr, i). Choose you preferred name ;-). [1] [4] if s.sub_equals(substr, i): # match ok What do you think? (bis) Thank you, Denis [1] I am unsure whether an end index is useful, actually I don't really understand its usage for startswith & endswith neither. [2] Actually, the compiler, if smart enough, may eliminate this object construction and just check the code; does it? Anyway, I think it is not that easy in the cases of ranges & sets. [3] As a side-note, 'ord' is in my view a misnomer, since character codes are not ordinals, with significant order, but nominals, plain numerical codes which only need to be all distinct; they are kinds of id's. For unicode, I call them 'ucodes', an idea I stole somewhere. But I would be happy is the method is called 'ord' anyway, since the term is established in the Python community. [4] Would such a new method make startswith & endswith unneeded?

15 73

Python-ideas December 2013