Mailman 3 February 2012 - Python-ideas

Re: [Python-ideas] doctest
by Michael Foord Feb. 28, 2012

Feb. 28, 2012

On 27 February 2012 23:23, Mark Janssen <dreamingforward(a)gmail.com> wrote: > On Mon, Feb 27, 2012 at 3:59 PM, Michael Foord <fuzzyman(a)gmail.com> wrote: >> >> As well as fundamental problems, the particular implementation of doctest >> suffers from these potentially resolvable problems: >> >> >>> Execution of an individual testing section continues after a failure. So >>> a single failure results in the *reporting* of potentially many failures. >>> >>> Hmm, perhaps I don't understand you. doctest reports how many failures > occur, without blocking on any single failure. > Right. But you typically group a bunch of actions into a single "test". If a doctest fails in an early action then every line after that will probably fail - a single test failure will cause multiple *reported* failures. > > >> The problem of being dependent on order of unorderable types (actually >>> very difficult to solve). >>> >> > Well, a crude solution is just to lift any output text that denotes an > non-ordered type and pass it through an "eval" operation. > Not a general solution - not all reprs are reversible (in fact very few are as a proportion of all objects). > > >> Things like shared fixtures and mocking become *harder* (although by no >>> means impossible) in a doctest environment. >>> >>> > This, I think, what I was suggesting with doctest "scoping" where the > execution environment is a matter of how nested the docstring is in > relation to the "python semantic environment", with a final scope of > "globs" that can be passed into the test environment, for anything with > global scope. > > >> Another thing I dislike is that it encourages a "test last" approach, as >>> by far the easiest way of generating doctests is to copy and paste from the >>> interactive interpreter. The alternative is lots of annoying typing of >>> '>>>' and '...', and as you're editing text and not code IDE support tends >>> to be worse (although this is a tooling issue and not a problem with >>> doctest itself). >>> >> > This is where I think the idea of having a test() built-in, like help(), > would really be nice. One could run test(myClass.mymethod) iterively while > one codes, encouraging TDD and writing tests *along with* your code. My > TDD sense says it couldn't get any better. > > >> More fundamental-ish problems: >> >> Putting debugging prints into a function can break a myriad of tests >> (because they're output based). >> > > That's a good point. But then it's a fairly simple matter of adding the > output device: 'print >> stderr, 'here I am'", another possibility, if TDD > were to become more of part of the language, is a special debug exception: > "raise Debug("Am at the test point, ", x)" Such special exceptions could > be caught and ignored by doctest. > > >> With multiple doctest blocks in a test file running an individual >> test can be difficult (impossible?). >> >> This again solved with the test() built-in an making TDD something that > is a feature of the language itself. > I don't fully follow you, but it shouldn't be hard to add this to doctest and see if it is really useful. > > >> I may be misremembering, but I think debugging support is also >> problematic because of the stdout redirection >> > > Interesting, I try to pre-conceive tests well enough so I never need to > invoke the debugger. > Heh. When I'm adding new features to existing code it is very common for me to write a test that drops into the debugger after setting up some state - and potentially using the test infrastructure (fixtures, django test client perhaps, etc). So not being able to run a single test or drop into a debugger puts the kybosh on that. Michael > > >> So yeah. Not a huge fan. >> >> That's good feedback. Thanks. > > Mark > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

4 3

Fwd: doctest
by Mark Janssen Feb. 28, 2012

Feb. 28, 2012

On Mon, Feb 27, 2012 at 12:58 PM, Ethan Furman <ethan(a)stoneleaf.us> wrote: > Mark Janssen wrote: >> Personally, I don't find unittest very suitable for test-driven >> *development*, although it *is* obviously well-suited for writing >> assurance tests otherwise. > > I like unittest for TDD. I should probably correct myself. It is suiltable, just not enjoyable. But now I know you are someone who likes all that arcana of unittest module. > unittest can be a bit bulky, but definitely worth it IMO, especially when > covering the corner cases. Corner cases are generally useful for the developer to know about, so its worth it to mention (==> test) in the documentation. > I have not used doctest, but I can say that I strongly dislike having more > than one or two examples in a docstring. This is often just a failure to separate tests property among different methods. > The other gripe I have (possibly easily fixed): my python prompt is '-->' > (makes email posting easier) -- should my doctests still use '>>>'? Will > doctest fail on my machine? As written, yes, but easily changeable in the module code for your unique case.... mark

2 2

Fwd: doctest
by Mark Janssen Feb. 28, 2012

Feb. 28, 2012

On Mon, Feb 20, 2012 at 11:28 AM, Barry Warsaw <barry(a)python.org> wrote: > On Feb 17, 2012, at 02:57 PM, Mark Janssen wrote: > FWIW, I think doctests are fantastic and I use them all the time. There are > IMO a couple of things to keep in mind: > > - doctests are documentation first. Specifically, they are testable > documentation. What better way to ensure that your documentation is > accurate and up-to-date? (And no, I do not generally find skew between the > code and the separate-file documentation.) > > - I personally dislike docstring doctests, and much prefer separate reST > documents. These have several advantages, such as the ability to inject > names into doctests globals (use with care though), and the ability to set > up the execution context for doctests (see below). The fact that it's so > easy to turn these into documentation with Sphinx is a huge win. > > Since so many people point this out, let me say that I completely agree that > doctests are not a *replacement* for unittests, but they are a fantastic > *complement* to unittests. When I TDD, I always start writing the > (testable) documentation first, because if I cannot explain the component > under test in clearly intelligible English, then I probably don't really > understand what it is I'm trying to write. > > My doctests usually describe mostly the good path through the API. > Occasionally I'll describe error modes if I think those are important for > understanding how to use the code. However, for all those fuzzy corner cases, > weird behaviors, bug fixes, etc., unittests are much better suited because > ensuring you've fixed these problems and don't regress in the future doesn't > help the narrative very much. I think is an example of (mal)adapting to an incomplete module, rather than fixing it. I think doctest can handle all the points you're making. See clarification pointers below... >>1. Execution context determined by outer-scope doctest defintions. > > Can you explain this one? I gave an example in a prior message on this thread, dated Feb 17. I think it's clear there but let me know. Basically, the idea is that since the class def can also have a docstring, where better would setup and teardown code go to provide the execution context of the inner method docstrings? Now the question: is it useful or appropriate to put setup and teardown code in a classdef docstring? Well, I think this requires a committment on the behalf of the coder/documentor to concoct useful (didactic) example that could go there. For example, (as in the prior-referenced message) I imagine putting example of defining a variable of the classes type (">>> g = Graph({some complex, interesting initialization})"), which might return a (testable) value upon creation. Now this could, logically, be put in the classes __init__ method, but that doesn't make sense for defining an execution context, and *in addition*, that can be saved for those complex corner cases you mentioned earlier. > I usually put all this in an additional_tests() method, such as: Yes, I do the same for my modules with doctests. A dummy function which can catch all the non-interesting tests. This, still superior, in my opinion, than unittest. It is easier syntactically, as well as for casual users of your code (It has no leaning curve like understanding unittest). This superiority to unittest, by the way, is only realized if the second suggestion (smart comparisons) is implemented into doctest. >>2. Smart Comparisons that will detect output of a non-ordered type >>(dict/set), lift and recast it and do a real comparison. > > I'm of mixed mind with these. Yes, you must be careful with ordering, but I > find it less readable to just sort() some dictionary output for example. What > I've found much more useful is to iterate over the sorted keys of a dictionary > and print the key/values pairs. Yes, but you see you're destroying the very intent and spirit of doctest. The point is to make literate documentation. If you adapt to it's incompleteness, you reduce the power of it. >>Without #1, "literate testing" becomes awash with re-defining re-used >>variables which, generally, also detracts from exact purpose of the >>test -- this creates testdoc noise and the docs become less useful. >>Without #2, "readable docs" nicely co-aligning with "testable docs" >>tends towards divergence. > > I've no doubt that doctests could be improved, but I actually find them quite > usable as is, with just a little bit of glue code to get it all hooked up. As > I say though, I'm biased against docstring doctests. Well, hopefully, I've convinced you a little that the limitations in doctests over unittests are almost, if not entirely due, to the incompleteness of the module. If the two items I mentioned were implemented I think it would be far superior to unittest. (Corner cases, etc can all find a place, because every corner case should be documented somewhere anyway!!) Cheers!! mark santa fe, nm

4 3

adding a Debug exception?
by Mark Janssen Feb. 28, 2012

Feb. 28, 2012

Had an idea on another thread (doctest) about a special exception called "Debug" that could could be raised to generate arbitrary output to stderr. This would be used instead of spurious print statements in code to inform developers during debugging (which might throw off doctest, for example). It could also replace "assert" (and improve upon it) which seems to be deprecated. Also, the __debug__ global could actually gain some functionality... Its "argument" could be an `eval`uatable string (checked at compile time) and it's output, this very string *plus* the output of it (if it evaluates to something different than itself). Just an idea.... mark santa fe

3 2

[Python-Dev] matrix operations on dict :)
by Mark Janssen Feb. 28, 2012

Feb. 28, 2012

Perhaps more specifically, I want to define a "grouping" (as encapsulated semantically and syntactically in a dict), at the place where the transition from atomic element into a group occurs. Syntactically this will be *denoted* by the curly brackets {}, operationally this will be defined in the CPython code itself. But the semantics in the middle must be "hashed out" for either to occur (pardon the pun). The question is whether it's premature to attempt such a task or not.... mark

1 0

Fwd: [Python-Dev] matrix operations on dict :)
by Mark Janssen Feb. 28, 2012

Feb. 28, 2012

More messages I didn't realize weren't being sent to the group....[mark] On Wed, Feb 8, 2012 at 7:13 PM, Stephen J. Turnbull <stephen(a)xemacs.org>wrote: > Mark Janssen writes: > > > The math (in my world) simply decided that factorial(0)=1 as the > > convention of "an empty product" (Wikipedia::Factorial). > > In modern math (ie, post-Eilenberg-Mac Lane), it's not really a > convention (unlike, say, Euclid's Parallel Postulate); it's the only > way to go if you want the idea of product to generalize. If you don't > understand that, I have serious doubts that you know what you're > talking about. If you do understand that, please take care to be more > precise. > Awesome. I didn't know anyone else really understood this kind of issue. Yes, I want the idea to generalize. In this case, not of "product" and arithmetic (in a mathematical space), but of "object model" and the notion of "grouping" (in a set-theoretical space). So a formalization must be made, and perhaps this arena will be the place to do that. I have to say that I'm approaching this from in the domain of computer science, so in some ways creating a definition in a new "space", or at least a space separate from the Platonian "Abstract" of mathematics. Love it! cheers! Mark

1 0

Support other dict types for type.__dict__
by Mark Janssen Feb. 28, 2012

Feb. 28, 2012

On Sat, Feb 25, 2012 at 4:05 PM, Steven D'Aprano <steve(a)pearwood.info> wrote: > Ned Batchelder wrote: > >> The Python answer for people who want read-only data structures has always >> been, "Don't modify them if you don't want to, and write docs that tell >> other people not to as well." What are you building that this answer isn't >> good enough? > > That is silly. That alleged "Python answer" is like telling people that they > don't need test frameworks or debuggers because the "Python answer" for > people wanting to debug their code is not to write buggy code in the first > place. Perhaps a good middle ground for this is to NOT tie it to particular data structures (like tuples vs lists), but abstract it by making an "immutable bit" that is part of the basic Object type. This doesn't give complete security, but does *force* a choice by a human agent to deliberately modify data. (This was actually going to be implemented in a sort of python fork several years ago.) There could be a "mutable?" check that returns True or False. mark Santa Fe, NM

8 13

Re: [Python-ideas] Fwd: doctest (and.... python3000)
by Ethan Furman Feb. 28, 2012

Feb. 28, 2012

Mark Janssen wrote: > On Mon, Feb 27, 2012 at 3:56 PM, Ethan Furman <ethan(a)stoneleaf.us > <mailto:ethan@stoneleaf.us>> wrote: > > As probably the easiest example, what is gained by having regression > tests as a document? With unittest you write a test with the > expected output and your done. I would imagine a doctest being > something like > > """This bug introduced in version 2.7.1, fixed in 2.7.2 > >>> this = quibble('that') > >>> this.attr > 'correct value' > """ > > > Huh? Perhaps I'm being dumb, but this is generally done outside of > unittest and within the code itself, something like: > > if sys.version > 2.4: > this= quibbleV4 > else: > this = quibbleV3 The choice of python similar version numbers was probably a mistake. The point is quibbleV3 has a bug in it, and I want to make sure that bug doesn't come back in later versions -- so I add a test in my unit tests to make sure that it doesn't. ~Ethan~

1 0

Re: [Python-ideas] Support other dict types for type.__dict__
by Victor Stinner Feb. 27, 2012

Feb. 27, 2012

Hum, after thinking twice, using a "frozendict" for type.__dict__ is maybe overkill for my needs (and intrused as noticed Nick). Attached patch for Python 3.3 is a simpler approach: add __final__ special value for class. If this variable is present, the type is constant. Example: --- class Test: __final__=True x = 1 Test.x = 2 # raise a TypeError Test.new_attr = 1 # raise a TypeError del Test.x # raise a TypeError --- There are various ways to deny the modification of a class attribute, but I don't know how to block the removal of an attribute of the addition of a new attribute without my patch. -- My patch is just a proof-of-concept. For example, it doesn't ensure that values are read-only too. By the way, how can I check that "a value is constant"? Except builtin immutable types, I suppose that the only way is to call hash(obj) and excepts an expect a TypeError. Victor

8 7

doctest
by Mark Janssen Feb. 27, 2012

Feb. 27, 2012

On Fri, Feb 17, 2012 at 10:08 PM, Steven D'Aprano <steve(a)pearwood.info> wrote: > Mark Janssen wrote: >> 1. Execution context determined by outer-scope doctest defintions. > > Can you give an example of how you would like this to work? > Sure, I wish I had a good example off the top of my head, but perhaps this will convey the idea: class MyClass(): """Yadda Yadda: foo's bars. >>> m = MyClass({some, sufficiently, interesting, initialization}) #tPOINT1: this variable (m) now accessible by all methods. "foobar check" #POINT2: possible output here is a useful test case not well-definable elsewhere without losing context. """ def method1(self, other): """Method method method method. >>> m.method("foo") #Now we see m is already defined and useable. "bar" """ def meth2(self, other): """Method to foo all bars >>> m.method("bar") #would have to decide whether a fresh m is redefined with each innerscope doctest (if we want side-effects to carry across inner doctests). (END) This is a basic example, sorry it's rather crude. There's probably a better example. (Think establishing a network socket connection or something in the class' doc which is then used by all the methods, for example.) >> 2. Smart Comparisons that will detect output of a non-ordered type >> (dict/set), lift and recast it and do a real comparison.> > > I would love to see a doctest directive that accepted differences in output > order, e.g. would match {1, 2, 3} and {3, 1, 2}. But I think that's a hard > problem to solve in the general case. I think this would be as simple as lifting the (string) output and doing an eval("{1,2,3}")=={3,2,1}, or (for security) using ast.literal_eval like Devin suggested. > I'd like a #3 as well: an abbreviated way to spell doctest directives, > because they invariably push my tests well past the 80 character mark. Hmm, seem like an alias could be defined easily enough, but I'll try to think about this when I have more time. mark

2 1