Mailman 3 February 2008 - pytest-dev

[py-dev] svn auth - revisiting svnwc auth object passing
by Guido Wesdorp Feb. 24, 2008

Feb. 24, 2008

Hi there! I've implemented auth support for py.path.svnurl now too, seems to work nicely. Since it requires authentication for just about all its methods, I decided (after some discussion with Holger on IRC and some playing around) to not have an 'auth' argument to all of the methods. Instead only the constructor groks an auth argument, path objects that are generated by any of the path's methods 'inherit' it, and an 'svnurl.auth' property is exposed to allow overriding it. It took a bit of playing around to get it to work properly, but I think it does now, please take a look if you're interested. If you like the way this works, I would (again ;) like to ask whether I'm allowed to change svnwc so it works the same there - at least, both for consistency's sake and because it's a bit cleaner imo - have an auth property instead of having the argument to all methods that can connect to the server would be nice... Please let me know what you think. Cheers, Guido

2 2

[py-dev] py.path.svnwc ensure problems
by Martijn Faassen Feb. 19, 2008

Feb. 19, 2008

Hi there, Imagine the following scenario: * we have a svnwc: wc * through some external mechanism, a directory 'map5' is created in it. This directory is *not* added to SVN using 'svn add' yet. * now through svnwc, we try to create a file in that directory by using .ensure(): wc.join('map5').join('content').ensure() We then get the following error: ... File "/home/faassen/buildout-eggs/py-0.9.0-py2.4.egg/py/path/svn/wccommand.py", line 160, in ensure p.add() File "/home/faassen/buildout-eggs/py-0.9.0-py2.4.egg/py/path/svn/wccommand.py", line 175, in add self._svn('add') File "/home/faassen/buildout-eggs/py-0.9.0-py2.4.egg/py/path/svn/wccommand.py", line 87, in _svn out = py.process.cmdexec(string) File "/home/faassen/buildout-eggs/py-0.9.0-py2.4.egg/py/process/cmdexec.py", line 94, in posix_exec_cmd ''.join(out), ''.join(err)) Error: ExecutionFailed: 1 LC_ALL=C svn add "/home/faassen/tmp/resynch/app2/data/map5/content" svn: '/home/faassen/tmp/resynch/app2/data/map5' is not a working copy svn: Can't open file '/home/faassen/tmp/resynch/app2/data/map5/.svn/entries': No such file or directory It would be nicer for my purposes if 'ensure()' also ensured that all directories above 'content' are added to SVN first (if indeed they haven't been added yet). Would this be good behavior in general for py.path.svnwc, or is there a good reason not to do this? Regards, Martijn

2 2

[py-dev] Svn auth update
by Guido Wesdorp Feb. 14, 2008

Feb. 14, 2008

Hi! I've implemented the SvnAuth class now, as Holger suggested, and made that the svnwc constructor, and the methods that connect to the server, all accept it as an argument (and use it, of course ;). You can see the progress on the 'guido-svn-auth' branch (SvnAuth code in py/path/svn/auth.py, and an updated version of the document I attached before in py/path/svn/auth.txt). After playing with the api a bit, I think agree with Holger that passing auth to children from methods that return path objects is not necessary, and may even be confusing, so perhaps we should indeed skip that for now. What I do want to add somehow is allow to set the --non-interactive switch somehow, although I'm not sure how (it doesn't seem proper to have it be part of the authentication information, but having yet another argument to all those methods is also not nice). I guess I'll continue with adding support for the auth argument to py.path.svnurl on friday, if no-one objects... Cheers, Guido

2 2

[py-dev] Some ideas
by Guido Wesdorp Feb. 12, 2008

Feb. 12, 2008

Hi! I thought a bit about the authentication feature for py.path.svn*, and wrote my ideas in a small doc. The doc is attached - please discuss if you're interested... Cheers, Guido SVN authentication support ========================== This document describes authentication support to both py.path.svnwc and py.path.svnurl (yet to be implemented). This should allow using the library in a completely automated situation, without having to provide the credentials interactively. Current ideas for implementation -------------------------------- The credentials are passed to the constructor of the path objects, and are used (transparently) for every action that accesses the server. Also, when provided, they are passed recursively to all child objects created by methods such as join(), ensure(), etc. It should be configurable in some way whether the credentials should be stored on disk. Storing them could be nice in certain situations (executive accesses to the repository will not require the credentials to be passed) but it might not be desired - for instance if a webserver runs more than one application, you would not want to pollute the webserver's home directory (if it even has one). This could be implemented by adding an additional argument to the constructors, too. Another addition that I would like to make to the API, which is required to deal properly with invalid credentials, is add another argument to the constructors, to specify whether the session should be interactive or not. This way we can specify whether we want to have the svn client ask for credentials when the provided credentials are invalid (the current default), or whether instead an exception should be raised. Code examples ------------- So, tying this together, code using this feature would look something like:: >>> wc = py.path.svnwc(url, username='user', password='pw', ... interactive=False, cache_auth=False) Open issues ----------- * How do we deal with externals properly? It looks like the svn command-line client uses the credentials provided for all externals, if possible, and either prompts for the password in interactive mode, or barfs when --non-interactive is passed. I think it makes sense to copy its behaviour here, pass the credentials to any child svn path objects (as discussed above), and either let the command-line app ask for creds or throw an exception when 'interactive' is set to False (see above). * Functional testing Functional testing is relatively hard, and will not work on all systems. It looks like a setup using 'svnserve' is possible, but it will be slow and provide relatively little advantages, apart from testing the integration. Holger suggested that perhaps functional tests could be skipped in favour of only unit tests. * Many new arguments required I think it's not very nice to have so many new arguments to the constructors, and especially dislike having an argument that's only useful when other args are provided (auth_cache). Not sure how to improve that, though...

2 3

[py-dev] 0.9.0:HEAD revision overview for 0.9.1 release
by Guido Wesdorp Feb. 9, 2008

Feb. 9, 2008

Hi! I went through all the checkins between r 38967 and the current HEAD and made lists of whether they were small bugfixes, refactorings and doc typos, or contain, or are part of, small or larger features. This hopefully helps when finding out what to port to the 0.9.1 branch for release... The document is attached. Cheers, Guido code ==== bugfixes/refactorings --------------------- * 39340 * 40002 * 40831 * 40832 * 40834 * 40934 * 41224 (? added failing test in code/testing/test_source.py) * 41480 * 43299 * 43575 * 44248 * 44648 * 44655 * 45294 * 45295 * 45483 * 45484 * 45518 * 45519 * 45535 * 45538 * 45514 * 45545 * 45547 * 45548 * 45549 * 45646 * 45647 * 45648 (? allow specifying reason for skips... does the rest depend on that?) * 45649 (? same as above) * 45655 * 45671 * 45901 * 45906 * 47277 * 49423 * 49974 * 50606 (already ported to dist in 50607) * 50645 * 50755 (ported from reporter-merge, already ported to dist in 50756) * 51285 * 51292 * 51392 (rename __package__ to __pkg__, already ported to bugfix-0.9.0 in 51330) merges ------ * 45539 (less threads for execnet) * 46010 (file locking in io) * 46011 (session cleanups) * 51033 (reporter-merge) * 51034 " small features -------------- * 39106 (+ to ALLOWED_CHARS) * 39655 (kill processes by PID on win) * 39982 (gateway._send) * 40702 (added setup/teardown for py.test generators) * 40737 (allow reporting FAILED TO LOAD MODULE as failure) * 40738 (FAILED TO LOAD MODULE reports in web browser) * 40739 (FAILED TO LOAD MODULE clickable) * 40753 (decrease MAX_COUNTER in webjs.py) * 41080 (filtering files in apigen) * 41620 (re-added source anchors to apigen) * 41655 (displaying exceptions in apigen) * 41855 (allow starting py.test from keyword test) * 41860 (kill start_on, add keyword_oneshot, simplifications) * 45994 (py.path.svnwc locking and non-recursive commits) * 46692 (py.path.svnurl 'export()' method) * 46770 (GeneratorExit exception) * 46771 " * 51092 " * 46772 (custom pdb.Pdb subclass that deals properly with py.code.Source) * 46794 " * 47584 (fixed '-k TestClass.test') * 48347 (added py.path.svnwc.status.replaced) * 50574 (mips and mipsel support for greenlet) not sure -------- * 42188 (reviving compat) * 42215 (reviving compat) larger features --------------- greenlet-based networking +++++++++++++++++++++++++ * 39937 * 39974 * 29975 * 39983 * 40705 * 40706 * 40719 * 40720 * 40982 * 41606 * 45550 * 51056 docs ==== typos, etc. ----------- * 38967 * 38969 * 39994 * 40001 larger changes -------------- * 39981 (ideas about pylib future) * 42085 (TODO) * 42089 (TODO) * 42092 (TODO) * 42103 (TODO) * 42191 (TODO) * 42193 (TODO) * 42194 (TODO) * 42273 (apigen ideas) * 45520 (planning) * 45525 (planning) * 51057 (1.0 TODO) * 51058 " * 51077 (bump version no. to 1.0-pre-alpha) * 51087 (note about broken skip-reporting after reporter-merge branch merge) * 51132 (TODO note about skip slowness)

1 0

[py-dev] __package__ attribute used by python 2.6
by Ralf Schmitt Feb. 8, 2008

Feb. 8, 2008

Hi all, __package__ is used extensively throughout the py lib. Unfortunately it will also be used by python 2.6 (see http://www.python.org/dev/peps/pep-0366/) I have replaced all occurences of __package__ with __pypackage__ to make py 0.9.0 work with python 2.6. I didn't run the testsuite but py.test seemed to work fine afterwards. Any plans to fix it? - Ralf

2 1

[py-dev] py.path.svn* username and password support
by Guido Wesdorp Feb. 7, 2008

Feb. 7, 2008

Hi! Someone asked me if I want to implement support for passing a username and password to the py.path.svn* functionality, allowing the application to provide a username and pw both for svnurls and svnwcs. I've investigated SVN and the code a bit, and it looks like this can be done in a relatively nice way: * svn seems to support command-line arguments for username and password, and also for telling it to not store auth information - with these arguments we can make sure the username/pw we provide are used, and not stored on the client * there's a switch --non-interactive which we can then enable to make very sure applications don't hang on asking username/password or things like that (currently that can be a problem when using svnwc or svnurl in scripts) I guess this would mean that for all operations that hit the server, the username and password arguments should be added. Perhaps it would be nice to have to provide it only on instantiation or something, although I fear that that will be harder to implement and use correctly in the situation of nested working copies (externals), etc. I hope you guys like the idea of having this in the py lib, it sounds like a useful addition to me... If no-one minds, I'll start implementing in a couple of days, I'd like to have it done (and tested well, obviously) before the release, if possible... Please let me know what you think. Cheers, Guido

2 1

[py-dev] Is py lib advertising 'initpkg' ?
by Brian Dorsey Feb. 5, 2008

Feb. 5, 2008

I ran into this comp.lang.python post today: http://groups.google.com/group/comp.lang.python/browse_thread/thread/5a9580… Seems like initpkg might be a good solution for the poster... however, I didn't see any reference to initpkg in any of the docs for py lib. Is it intended to be an implementation detail, or a tool for others to use? Take care, -Brian

2 1

[py-dev] py.execnet: hang in channel.receive()
by Bob Kuehne Feb. 1, 2008

Feb. 1, 2008

hi gang, i first saw holger present py.execnet at pycon a few years back, and have been waiting to use this cool tool. so now i'm on it, and i've got a problem. i'm running this on two macs, both running os x 10.5.1. i've installed py via easy_install, so i've got a luverly egg, and here's my script that i'm trying to just test with: -- import py gw = py.execnet.SshGateway( "localhost" ) channel = gw.remote_exec( 'channel.send( [ [ False, True, False, 0, ] ] )' ) print channel.receive() channel.close() gw.exit() -- so, on mac numero 1, running python 2.5.1, i see the output i'd expect: -- Python 2.5.1 (r251:54863, Oct 5 2007, 21:08:09) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import py >>> gw = py.execnet.SshGateway( "localhost" ) >>> channel = gw.remote_exec( 'channel.send( [ [ False, True, False, 0, ] ] )' ) >>> print channel.receive() [[False, True, False, 0]] >>> channel.close() >>> gw.exit() -- but on another mac, also python 2.5.1, i see the same, but the 'channel.receive()' is hung. if i break there (control-c), i get the following: -- Python 2.5.1 (r251:54863, Oct 9 2007, 20:39:46) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import py >>> gw = py.execnet.SshGateway( "localhost" ) >>> channel = gw.remote_exec( 'channel.send( [ [ False, True, False, 0, ] ] )' ) >>> print channel.receive() ^CKilled by signal 2. Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Library/Python/2.5/site-packages/py-0.9.0-py2.5.egg/py/ execnet/channel.py", line 172, in receive x = queue.get() File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/Queue.py", line 165, in get self.not_empty.wait() File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/threading.py", line 214, in wait waiter.acquire() KeyboardInterrupt -- now, the question is what's different. and it's at this point that i wave my arms and go, wtf? i really don't know, but i'm wondering aloud if anyone else has seen this sort of behavior, and if so, what they might have encountered or changed to get it to be better behaved. thanks, bob

2 1

[py-dev] "event" branch and architecture
by holger krekel Feb. 1, 2008

Feb. 1, 2008

Hi py-dev! i am currently hammering on the svn/py/branch/event branch [*]. The main intention is to modify py.test's internal architecture towards using events and to base reporting and some internal processing on these events. Here is the basic event loop (exposed as py.event.Hub for now): class Hub(list): """ General Event Hub """ def notify(self, event): for subscriber in self: subscriber(event) This is inspired by a talk from Jim Fulton about zope3's event loop - zope.event just contains a global list and a notify function like the above. However, i'd like to keep things more local for now (going global is usually much easier than going local, to begin with and local state is also nicer for TDD) For py.test the upcoming event archicture should allow us to e.g.: * have co-existing reporters e.g. one that writes test results to an internal database and one that provides terminal reporting. * Or one terminal and one html-output reporter. * GUI based reporters (where we can easily route events to GUI loops) * Or a small plugin that allows to automatically copy failure information to e.g. http://paste.pocoo.org/. * ... I am also using the event branch to do various cleanups and refactorings, mainly i am aiming at refactoring distributed and remote testing to share more code (and Events) with local testing. I hope that in the end one can write a reporter that does not need to care (too much) how tests are/were distributed across machines. best & cheers, holger [*] In fact i am working there in py/test2 - which is a branch itself of py/test. I did this (with the help of the py/bin/_maketest2.py helper, btw) in order to be able to use py.test for refactoring py.test. Otherwise if i change files in py/test/ and try to run py.test it would affect my very own test run, making TDD hard or impossible. -- Holger Krekel - freelance manager and programmer merlinux GmbH: http://merlinux.de py.test: http://pytest.org py lib: http://pylib.org PyPy: http://codespeak.net/pypy

2 2