Mailman 3 January 2005 - pytest-dev

[py-dev] py.path.dict and py.path.proxy patch
by Matthew Scott Feb. 14, 2005

Feb. 14, 2005

Round 2! The attached patch provides the new classes py.path.dict and py.path.proxy, as well as py.__impl__.path.virtual.virtual.Path which is the basis for them. This is an attempt at taking into account some of the suggestions that holger made regarding generalization of the py.path.embed patch I posted in the last couple of days. This patch supersedes the embed1.patch and embed1-1.patch files, so don't use those anymore. What py.path.proxy allows you to do is to take an arbitrary path and create a proxy filesystem into a limited area of that path's filesystem. Maybe a better word would be sandbox, but I found proxy to be easier to type. Naming suggestions are welcome. This code is set in fresh mud, not in stone. :) I haven't yet tried py.path.proxy using anything but py.path.local root paths. Here's an example of using py.path.proxy to wrap a directory specified by py.path.local, and of using py.path.dict to create a dictionary-based virtual file system that contains a copy of that directory: I started IPython in a directory underneath my home directory that contains pictures of a model of laptop that I owned years ago, the Toshiba T1200XE. I created a 'here' variable to contain that path. >>> import py >>> here = py.path.local() >>> here --- local('/home/gldnspud/personal/www/crap/t1200xe') Listing the directory, you can see that it has three files inside it: >>> here.listdir() --- [local('/home/gldnspud/personal/www/crap/t1200xe/1.jpg'), local('/home/gldnspud/personal/www/crap/t1200xe/2.jpg'), local('/home/gldnspud/personal/www/crap/t1200xe/3.jpg')] Now for the fun part. I created a 'prox' variable which is a py.path.proxy whose root is the 'here' path. >>> prox = py.path.proxy(root=here) >>> prox --- proxypath('/', local('/home/gldnspud/personal/www/crap/t1200xe')) If you list its directory, it obviously contains the same files: >>> prox.listdir() --- [proxypath('/1.jpg', local('/home/gldnspud/personal/www/crap/t1200xe')), proxypath('/2.jpg', local('/home/gldnspud/personal/www/crap/t1200xe')), proxypath('/3.jpg', local('/home/gldnspud/personal/www/crap/t1200xe'))] If you traverse to parent directories with 'here', it goes up and up and up. >>> here.join('..', '..') --- local('/home/gldnspud/personal/www') However, if you attempt the same thing using 'prox', you run into the boundary that was defined when prox's virtual filesystem was created. >>> prox.join('..', '..') --- proxypath('/', local('/home/gldnspud/personal/www/crap/t1200xe')) Now I'll create a 'virt' variable which will contain a py.path.dict starting from an empty dictionary. >>> virt = py.path.dict() >>> virt --- dictpath('/', <dict at 4064ed74>) >>> virt.listdir() --- [] I'll populate it with the contents of 'prox'. >>> prox.copy(target=virt) >>> virt.listdir() --- [dictpath('/2.jpg', <dict at 4064ed74>), dictpath('/3.jpg', <dict at 4064ed74>), dictpath('/1.jpg', <dict at 4064ed74>)] What's fun (but maybe pointless other than to show that when it isn't pointless to do so, it can be done) is creating a proxy for a dict path, and then a proxy for that proxy. >>> virt.join('a', 'b', 'c').ensure(dir=True) --- dictpath('/a/b/c', <dict at 4064ed74>) >>> virt.listdir() --- [dictpath('/2.jpg', <dict at 4064ed74>), dictpath('/a', <dict at 4064ed74>), dictpath('/3.jpg', <dict at 4064ed74>), dictpath('/1.jpg', <dict at 4064ed74>)] >>> prox2 = py.path.proxy(root=virt.join('a')) >>> prox2.listdir() --- [proxypath('/b', dictpath('/a', <dict at 4064ed74>))] >>> prox3 = py.path.proxy(root=prox2.join('b')) >>> prox3.listdir() --- [proxypath('/c', proxypath('/b', dictpath('/a', <dict at 4064ed74>)))] Removing the "c" directory in prox3 of course will result in a/b/c being removed from virt. >>> prox3.join('c').remove() >>> virt.join('a', 'b').listdir() --- [] Let's delete the jpg files from virt and create a small text file, and see what the underlying dictionary representation of virt is. (Hint: a 'c' key stands for contents. A directory's contents is a dictionary, and a file's contents is a string). >>> for path in virt.listdir(fil='*.jpg'): path.remove() ... >>> virt.listdir() --- [dictpath('/a', <dict at 4064ed74>)] >>> virt.join('test.txt').ensure().write('hello world') >>> virt.join('test.txt').read() --- 'hello world' >>> virt.fs.dict --- {'c': {'a': {'c': {'b': {'c': {}}}}, 'test.txt': {'c': 'hello world'}}} Fun stuff! - Matthew

2 1

[py-dev] py.path.embed patch
by Matthew Scott Jan. 29, 2005

Jan. 29, 2005

Here is the first stab at py.path.embed Let me know what you all think. If it passes muster, feel free to merge it into trunk. - Matthew

3 6

[py-dev] we are about to get press:
by Laura Creighton Jan. 28, 2005

Jan. 28, 2005

http://agiletesting.blogspot.com/2005/01/python-unit-testing-part-1-unittes… which got a pointer in the daily python url. woo woo woo! laura

5 5

[py-dev] Strange failure with py.test - too much magic?
by Paul Moore Jan. 28, 2005

Jan. 28, 2005

I thought I'd give py.test a try. So I grabbed the code from the Subversion repository and wrote a little test file, which just did an assert 1=0. I don't expect the test to succeed :-) I added the dist-py directory to PYTHONPATH, and ran dist-py\py\bin\pytest.cmd test_pytest.py. And got the attached result. What does all that mean?!?! I expected something a bit more straightforward. I'm running on Windows XP Pro, with Python 2.4 and py.test from Subversion. Paul. PS Is it possible to use py.test apart from the rest of the py library? I'm not interested in the XML stuff, or execnet, and while greenlets sound cool, I don't want to package them up with my testing framework...

6 8

[py-dev] thoughts on embedding files
by Matthew Scott Jan. 27, 2005

Jan. 27, 2005

(cross-posting this to both the Schevo and Py mailing lists, to get the input of both sets of people) Extending evo ============= The next task I am embarking upon for the Schevo project is to add two new actions to evo, a tool that was just added that is a central place to perform actions to create, run, and generally work with Schevo applications: * "evo py2exe", which will perform all necessary steps to turn your application into a Windows executable using Py2exe. * "evo innosetup", which will perform the py2exe action, then wrap the executable inside a Windows installer using the free InnoSetup program. (see http://lists.orbtech.com/pipermail/schevo-devel/2005-January/000060.html for more information) These actions will be built to satisfy the requirements of an application that Patrick and I are working on professionally, but will be useful for packaging any Schevo application for deployment on Windows platforms. In the future, expect to see "evo cxfreeze", and perhaps one day "evo deb" and "evo rpm". Embedding files --------------- One of the requirements of turning a Schevo application into a Py2exe application is that of file embedding. If you have looked at Schevo prior to the latest merge, you might have noticed that each application had its own "build.py" step. This step did the following things: * Embedded the contents of the schema directory in a single Python module. * Determined the icons that the application used, and the icon collections that are made available by the application, and embedded those icons in a single Python module. * For some apps, embedded arbitrary files that the user interface would use, such as additional graphics other than icons. The reason all of this was necessary is that we wanted to make sure that the entirety of the schema, and all of the icons an app would use, would be embedded within a Py2exe application's library ZIP file, and not just copied into plain files that could be easily modified by a user of the application. The easy way to do this during development, of course, is to always build the generated Python modules, and to always use those modules. This worked in the short term, but made application startup during development time-consuming, and added much boilerplate necessary for creating new applications. What we want to do instead is to make sure that when developing a Schevo application, the developer does NOT worry about how or when to embed files in this manner. The developer should just be able to create the application, efficiently run it in a development state using normal files, and then run "evo py2exe" to create an executable. The "evo py2exe" action should embody all of the logic needed to determine what needs to be embedded and where, and the final application that is generated should have the necessary logic to know that it should read file-like objects from the embedded Python modules rather than real files on disk. So, it becomes necessary to create an API to allow this transparency to occur, so that the application developer must only make minimal changes to his/her way of thinking but may still take advantage of files embedded within Python modules. The py.path API --------------- Early on in the Schevo project after it was split from Pypersyst, we made the decision to use py.test from the "py" library to run our unit tests. We were happy with this decision, but this meant that the user would need to keep up-to-date with the py library, which is under heavy development. So we kept our usage of py within Schevo optional, and minimal. Now that we are including py within Schevo as a dependency, we can control which revision of it we use. So if some major API shift occurs in the main py distribution, we can just stick with a "known good" revision for Schevo. Because of this, we are moving from tapping the water lightly with our toes to walking into it knee deep. :) The py.path API is the first major step toward using py for more than just unit testing. py.path provides a very well-designed object model for dealing with paths of all sorts, both local, remote, and virtual. I've started using it in Schevo whenever I refactor a piece of code that works with filesystem paths. So, what I am thinking is that a py.path.embed package could be created that would embody the aspects of embedding files within a Python module, along with providing a single API that could be used to access those files both when they are still in the local filesystem, and when they have been embedded into Python modules. I will be writing more as I discover more about py.path's innards, and determine whether this would be a good choice or not for embedding files. Any feedback from the Py and Schevo teams is most appreciated :)

2 2

[py-dev] Problems with DOS files
by Yaroslav Bulatov Jan. 19, 2005

Jan. 19, 2005

py.test runs into problems with files using DOS linebreaks (ie \r\n) under Cygwin When I run py.test on a file with DOS linebreaks I get something like def test_5(): ^ SyntaxError: invalid syntax On other hand, when I run the same file using the same python executable, it reports no error. $ /usr/bin/python2.3 test_maxent_classifier.py Traceback (most recent call last): File "test_maxent_classifier.py", line 4, in ? if __name__=='__main__': test_5() File "test_maxent_classifier.py", line 2, in test_5 assert 1==2 AssertionError $ py.test============================= test process starts ============================= testing-mode: inprocess executable : /usr/bin/python2.3 (2.3.4-final-0) =============================================================================== C.. _______________________________________________________________________________ ________________________________ Collect Error ________________________________ def getpycodeobj(self): dotpy = self.check(ext='.py') if dotpy: my_magic = py.std.imp.get_magic() my_timestamp = int(self.mtime()) if __debug__: pycfile = self + 'c' else: pycfile = self + 'o' try: f = pycfile.open('rb') try: header = f.read(8) if len(header) == 8: magic, timestamp = py.std.struct.unpack('<4si', header) if magic == my_magic and timestamp == my_timestamp: return py.std.marshal.load(f) finally: f.close() except IOError: pass s = self.read() > codeobj = compile(s, str(self), 'exec') > /cygdrive/c/Python23/Lib/site-packages/dist-py/py/path/local/local.py, line 36 0 File "/cygdrive/g/research/maxent/classifier/test_maxent_classifier.py", line 1 def test_5(): ^ SyntaxError: invalid syntax ================== tests finished: 2 passed in 0.05 seconds ================== On other hand, everything works fine if I convert the files to unix mode

13 25

[py-dev] Issue with py.test and stderr
by Grig Gheorghiu Jan. 17, 2005

Jan. 17, 2005

Hi, Holger I started to play with py.test after seeing it mentioned on a lot of Python-related blogs. I find it really easy to use and powerful at the same time, so congrats for a fine piece of work. I did run into a slight issue though. At some point I was instantiating a class which was redirecting stderr to stdout under some circumstances. The __del__ method of my class was resetting sys.stderr to sys.__stderr__. I think this caused a problem with py.test, in that it interfered with py.test's own manipulation of sys.stderr. The end result was that I was getting a rather cryptic error message from py.test: # py.test -v test_clients.py ============================= test process starts ============================= testing-mode: inprocess executable : /usr/bin/python (2.2.3-final-0) using py lib: /local/dist-py/py <rev 8277> initial testconfig 0: /local/dist-py/py/test/defaultconfig =============================================================================== Traceback (most recent call last): File "/local/dist-py/py/bin/py.test", line 5, in ? main() File "/local/dist-py/py/test/cmdline.py", line 32, in main run.inprocess(args, filenames) File "/local/dist-py/py/test/run.py", line 141, in inprocess driver.run(fncollectors) File "/local/dist-py/py/test/drive.py", line 43, in run self.run_collector_or_item(x) File "/local/dist-py/py/test/drive.py", line 61, in run_collector_or_item self.runcollector(obj) File "/local/dist-py/py/test/drive.py", line 83, in runcollector self.run_collector_or_item(obj) File "/local/dist-py/py/test/drive.py", line 61, in run_collector_or_item self.runcollector(obj) File "/local/dist-py/py/test/drive.py", line 83, in runcollector self.run_collector_or_item(obj) File "/local/dist-py/py/test/drive.py", line 59, in run_collector_or_item self.runitem(obj) File "/local/dist-py/py/test/drive.py", line 103, in runitem self.reporter.enditem(res) File "/local/dist-py/py/test/report/text/reporter.py", line 141, in enditem result.out, result.err = item.iocapture.reset() File "/local/dist-py/py/test/tool/outerrcapture.py", line 29, in reset err = e.getvalue() AttributeError: 'file' object has no attribute 'getvalue' The only way I was able to get around this issue without modifying my class was to edit py/test/tool/outerrcapture.py and replace: err = e.getvalue() with try: err = e.getvalue() except: err = None Then things seemed to work fine. The real solution I found was to modify my class and save sys.stderr into a temp and restore it in __del__ to that temp, instead of restoring it to sys.__stderr__. Then I could get rid of my modifications of outerrcapture.py. Anyway, I guess this is a pretty rare case, but maybe you can insert a try/catch in outerrcapture.py to prevent things like this from happening. All the best, Grig P.S. I'm really interested in both Python and testing, so if you guys need some help with py.test in the future, I'd be glad to pitch in. I have a blog with some of the stuff I'm doing at http://agiletesting.blogspot.com . I've been using unittest quite heavily, but I'm also trying to look into alternatives, mainly py.test and doctest. I released a JUnitPerf port to Python which I called pyUnitPerf, so now I'm thinking about doing something similar and more "pythonic" with py.test. Maybe py.test.perf?!!

5 8

[py-dev] Re: [Python-Dev] Getting rid of unbound methods: patch available
by Armin Rigo Jan. 17, 2005

Jan. 17, 2005

Hi Holger & py.testers, On Sun, Jan 16, 2005 at 10:12:37PM -0800, Guido van Rossum wrote: > https://sourceforge.net/tracker/index.php?func=detail&aid=1103689&group_id=… > > Here's a patch that gets rid of unbound methods, as > discussed here before. A function's __get__ method > now returns the function unchanged when called without > an instance, instead of returning an unbound method object. This patch breaks py.test quite extensively, which is no surprise because it contains a number of tests along the lines of hasattr(func, 'im_self'). After a quick scan over py.test I'm not sure how easily this could be fixed, and more importantly how easily it would be to fix it while still preserving compatibility with existing Python versions. The "hard" bit with removed unbound methods is that it's no longer possible to find back the class C from which a function C.f was read (i.e. there is no im_class any more). I would suggest that we (I) give the issue some serious thoughts, and if it really makes the life of the py lib difficult, it will be a good "real life" example of breakage. I'm sure it would be a good argument to postpone the patch to the faraway ages usually called "Python 3000". Armin

2 3

[py-dev] improved reporting ...
by hpk＠trillke.net Jan. 17, 2005

Jan. 17, 2005

hi py.test.onistas, i have cleaned up and improved the text-reporting for failing tests (i think :-). You can see the results with e.g. py.test py/documentation/example/pytest/failure_demo.py which gives you a list of 40 reported failing tests. have fun (and report back glitches), holger

3 2

[py-dev] Re: [py-svn] r8187 - py/dist/py
by hpk＠trillke.net Jan. 17, 2005

Jan. 17, 2005

Hi Armin, On Sun, Jan 09, 2005 at 19:28 +0100, arigo(a)codespeak.net wrote: > Author: arigo > Date: Sun Jan 9 19:28:38 2005 > New Revision: 8187 > > Modified: > py/dist/py/initpkg.py > Log: > Temporary fix for "import py; py.magic.greenlet" raising an ImportError if the > 'py' lib is not in your PYTHONPATH: make the package __path__ absolute. This > prevents os.chdir() from breaking imports. I see. And also saw your IRC ramblings as i just got back. Indeed, os.chdir() can break things horribly for relatively imported packages. As you hinted at, it is likely a problem with Python itself that it doesn't try to convert import paths to absolute locations as soon as possible. Of course, the py lib's import hook may very well obscure things further although (arguably) the errors from Python when just using plain imports are very similar if you don't realise that your import-problem happens while executing in some os.chdir()ed execution scope. OTOH, the specific problem with make_module_from_c() might not have arisen because we might have imported py.code.Source during module initilization. OTOH, it might have occured within a C-file that imports python-modules during module initilization :-) Anyway, thanks for figuring this out and fixing it. I hope it wasn't as hard as fixing some Psyco problems :-) cheers, holger

3 2