Mailman 3 February 2005 - pypy-dev

Some more tests passing
by Seo Sanghyeon 05 Feb '05

05 Feb '05

As of r8877, some more CPython regression tests are passing, unmodified. test_textwrap is passing in 393 seconds. test_difflib is passing in 205 seconds. test_pprint is passing in 318 seconds. Probably more. For comparison, my machine benchmarks at 21276 pystones. Why don't you reply to this message, if you got some more tests passing? Seo Sanghyeon

3 3

datetime module
by David Fraser 04 Feb '05

04 Feb '05

Hi I have been testing out pypy for the first time... I wanted to get the datetime module working and found that there was a pure python implementation of it done before the C implementation, in the nondist/sandbox/datetime module on Python CVS. This implementation works fairly well on pypy: - in order to test it, the attached patch to test_datetime is needed to avoid problems with pickle and gc - other than pickling, the only test failures are below The datetime.py can be got directly here: http://cvs.sourceforge.net/viewcvs.py/*checkout*/python/python/nondist/sand… ====================================================================== ERROR: test_combine (__main__.TestDateTime) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_datetime.py", line 1396, in test_combine dt = combine(time=t, date=d) TypeError: newfunc() got 2 unexpected keyword arguments ====================================================================== ERROR: test_more_astimezone (__main__.TestDateTimeTZ) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_datetime.py", line 2636, in test_more_astimezone dt = self.theclass.now(tz=f44m) TypeError: newfunc() got an unexpected keyword argument 'tz' ====================================================================== ERROR: test_tzinfo_fromtimestamp (__main__.TestDateTimeTZ) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_datetime.py", line 2423, in test_tzinfo_fromtimestamp again = meth(ts, tz=off42) TypeError: newfunc() got an unexpected keyword argument 'tz' ====================================================================== ERROR: test_tzinfo_now (__main__.TestDateTimeTZ) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_datetime.py", line 2385, in test_tzinfo_now again = meth(tz=off42) TypeError: newfunc() got an unexpected keyword argument 'tz' ====================================================================== FAIL: test_subclass_timedelta (__main__.TestTimeDelta) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_datetime.py", line 469, in test_subclass_timedelta self.assert_(type(t3) is timedelta) File "/share/sjsoft/code/Python/pypy/dist-pypy/lib-python-2.3.4/unittest.py", line 278, in failUnless if not expr: raise self.failureException, msg AssertionError David

3 7

Hi!
by Carl Friedrich Bolz 04 Feb '05

04 Feb '05

Hi pypy-list! I've [1] been following the pypy-dev mailing list for quite some time now and am really exited about this project. This weekend I checked out the code and started to play around with it a bit. Since there has been some talk about adding a LLVM backend and since this doesn't seem to have happened I decided to take a stab at it. I installed LLVM (which really is a pain), read the LLVM documentation and started to write a (very rudimentary) genllvm.py. It can already generate LLVM-assembler for simple functions (e.g. just ints, no function calls, no default arguments...). Then a Pyrex-wrapper for the functions is generated so that they can imported. For the function snippet.my_gcd the following LLVM-assembler code is generated: int %my_gcd(int %a_2, int %b_3) { block0: %r_7 = call int %mod(int %a_2, int %b_3) br label %block1 block3: %a_29 = phi int [%a_8, %block1] %b_30 = phi int [%b_9, %block1] %r_31 = phi int [%r_10, %block1] %v32 = phi bool [%v11, %block1] %r_21 = call int %mod(int %b_30, int %r_31) br label %block1 block2: %v4 = phi int [%b_9, %block1] ret int %v4 block1: %a_8 = phi int [%a_2, %block0], [%b_30, %block3] %b_9 = phi int [%b_3, %block0], [%r_31, %block3] %r_10 = phi int [%r_7, %block0], [%r_21, %block3] %v11 = call bool %is_true(int %r_10) br bool %v11, label %block3, label %block2 } Note that this exactly mirrors the flowgraph. I just use function calls for all SpaceOperations (though some probably have to be special-cased later). It is not neccessary to rename these functions since LLVM considers functions to be different if their signatures differ. The implementation of these functions is: int %mod(int %a, int %b) { %r = rem int %a, %b ret int %r } bool %is_true(int %a) { %b = cast int %a to bool ret bool %b } LLVM omptimizes the above code to: int %my_gcd(int %a_2, int %b_3) { block0: %r.i = rem int %a_2, %b_3 br label %block1 block3: %r.i1 = rem int %b_9, %r_10 br label %block1 block2: ret int %b_9 block1: %b_9 = phi int [ %b_3, %block0 ], [ %r_10, %block3 ] %r_10 = phi int [ %r.i, %block0 ], [ %r.i1, %block3 ] %b.i = seteq int %r_10, 0 br bool %b.i, label %block2, label %block3 } This is then compiled to native code. At the moment I'm not using the LLVM-API to generate this code since it was simpler to just do the string shuffling in python than to wrap and learn the LLVM-API. In my opinion this can be extended to nearly all of Python's data types as long as the annotation succeeds. I cannot yet judge wether other things like classes, exception handling, garbage collection etc. will be easy but we shall see. As for the code: It is quite convoluted and ad-hoc, I need to clean it up, write some more tests (I already wrote some) and extend it a bit before it is fit for someone else to see. Should I just post it or apply for checkin rights? What do you all think? Does my approach make sense or are there some obvious problems that I didn't see. Regards, Carl Friedrich [1] To introduce myself shorly: My name is Carl Friedrich Bolz, I'm 21 and studying maths and physics in Heidelberg, Germany, currently in my 3rd semester. I've been using Python since four years, mostly for my own projects. In addition I did/am doing a bit of C/C++ programming, mostly for 3D-graphics and high energy physics data analysis.

4 4

Re: [pypy-dev] Hi!
by Laura Creighton 04 Feb '05

04 Feb '05

In a message of Thu, 03 Feb 2005 15:00:36 +0100, "Carl Friedrich Bolz" writes: <snip> >I won't be able to go to PyCon but I hope I can make it to one of the >sprints after that at PyCon. Is there some sprint planning that goes a >bit beyond just the next one? I'm even thinking about organizing a >sprint here in Heidelberg this summer/fall though I don't know whether >this will fly. <snip> Plan for a Sprint after ACCU/PythonUK in April in Oxford, and after EuroPython in Göteborg, Sweden. exact dates not set yet. I am working on setting these up. Laura x

1 0

xrange object start/stop/step attribute
by Seo Sanghyeon 03 Feb '05

03 Feb '05

I wanted to confirm that PyPy's applevel tests are indeed passing in CPython, and discovered this: pypy/module/test/test_builtin.py test_xrange_args tests start/stop/step attribute of xrange object. But these attributes got deprecated in 2.2, and is not present in 2.3 or 2.4... Well, that test is still useful though. By the way, is there already a tool to run applevel tests on CPython? Seo Sanghyeon

2 1

even more flow probs
by Christian Tismer 03 Feb '05

03 Feb '05

Hi Armin, I undid all my patching, but patched _classobj to have no **kwds any longer. After that, flow ran infinitely on this file. Virtual mem grew to over 500 MB. After 15 minutes, I interrupted the process. The traceback was: """ <function __call__ at 0x01501DB0> Traceback (most recent call last): File "D:\Python23\lib\site-packages\Pythonwin\pywin\framework\scriptutils.py", line 310, in RunScript exec codeObject in __main__.__dict__ File "D:\pypy\dist\pypy\translator\geninterplevel.py", line 1333, in ? gen.gen_source(fname, ftmpname) File "D:\pypy\dist\pypy\translator\geninterplevel.py", line 725, in gen_source self.gen_source_temp() File "D:\pypy\dist\pypy\translator\geninterplevel.py", line 788, in gen_source_temp self.gen_rpyfunction(func) File "D:\pypy\dist\pypy\translator\geninterplevel.py", line 842, in gen_rpyfunction body = list(self.rpyfunction_body(func, locals)) File "D:\pypy\dist\pypy\translator\geninterplevel.py", line 959, in rpyfunction_body SpaceClass=AugmentedFlowSpace) File "D:\pypy\dist\pypy\translator\translator.py", line 77, in getflowgraph graph = space.build_flow(func) File "D:\pypy\dist\pypy\objspace\flow\objspace.py", line 174, in build_flow ec.build_flow() File "D:\pypy\dist\pypy\objspace\flow\flowcontext.py", line 208, in build_flow block.patchframe(frame, self) File "D:\pypy\dist\pypy\objspace\flow\flowcontext.py", line 43, in patchframe replaylist) File "D:\pypy\dist\pypy\objspace\flow\flowcontext.py", line 49, in __init__ def __init__(self, listtoreplay, nextblock, booloutcome, nextreplaylist): KeyboardInterrupt >>> """ HEEEEEEELP, please! -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

1 0

Flow problems
by Christian Tismer 03 Feb '05

03 Feb '05

Hi Armin, I tried to generate old-style classes interp-level. First problem: flow space has no w_dict. Well, I did a surrogate. As a follow-up, augmented flow-space barfed at unpackiterable without explicit length. I tried harder and undefined flow's unpackiterable. Furthermore, I had to add w_TypeError. This both was for sure bad, since now default's unpackiterable was in place, and in turn, flow tried to unwrap a non- -constant, causing File "D:\pypy\dist\pypy\objspace\flow\objspace.py", line 128, in unwrap raise UnwrapException So, in a way, the things I'm doing do not make sense. In a way, using flow for app2interp, poses extra demands on flow for which it was not designed. The question is, how hard is it to really add var-keywords for this case? It is necessary, and I would love to add it, but I don' understand what goes wrong, here. Thanks for advice -- chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

1 0

Re: [pypy-svn] r8804 - in pypy/dist: lib-python-2.3.4 pypy/lib/test2 pypy/module
by hpk＠trillke.net 03 Feb '05

03 Feb '05

Hi Adim, On Wed, Feb 02, 2005 at 14:08 +0100, adim(a)codespeak.net wrote: > Author: adim > Date: Wed Feb 2 14:08:26 2005 > New Revision: 8804 > > Added: > pypy/dist/lib-python-2.3.4/itertools.py Added or modified modules should go to pypy/lib. Basically we want to keep the lib-python tree as clean as possible (only conftest.py and pypy_unittest.py are there as mentioned in Samuele's Leysin-Report mail). cheers, holger

2 1

Summary of Leysin Sprint
by Samuele Pedroni 02 Feb '05

02 Feb '05

Hello, This is a summary of what was accomplished and worked on during last sprint in Leysin. Leysin is a village in the Swiss Alps, home to Armin Rigo. The views of the surrounding mountains were spectacular. It seemed to inspire us to work harder. Thanks to Armin for organizing it: * All project participants met and worked together for the first time. We introduced newcomers to the project to the PyPy code base and the development process. We gave introductions to the basic functioning and the structure of the code base. py.test our unit testing framework was introduced to those who hadn't seen it before. * We also got to know our Process Manager, Bea Düring from Changemaker. Bea will produce her own report. If you don't know the difference between 'project management', 'people management', and 'process management' then you probably would enjoy learning the differences from Bea. Bea interviewed the sprint participants to find out about their expectations, dissemination and more general ideas about the PyPy project and where it should go in the future. This Sprint went extremely well, but future ones should go even better, since we are learning how to improve all the time. Bea is making a report that discusses this more. * We made plans for new and improved infrastructure. Thank you svn, we can change our repository layout whenever we feel it would be a good idea. :-) But as for technical changes to pypy itself: * The flow object space now uses variable names derived from the original python ones and propagate them as possible. (Armin, Olivier Dormond) * The typedunwrap branch has been merged back to the trunk. (Armin, Samuele) Type specific unwraps (int_w, str_w, ...) should be used everywhere possible. The type generic unwrap should be used only in tests, at bootstrap time and or temporarily in cases where we are borrowing heavily from underlying CPython, because it is annotator unfriendly and is able to unwrap mixed types containers which are not reasonably RPythonic. * Updated documentation to describe new type specific unwrap operations. (Michael, Samuele) * We can now import and run the CPython tests on top of PyPy. This found lots of small problems, and a few not so small ones. So we made lots of small fixes, and implemented missing stuff, which was a good way for newcomers to get to know the codebase. We added or completed functionality to sys module: - _getframe() - set/getrecursionlevel() - exc_clear() ... (Adrien Di Mascio, Anders Chrigström, Armin, Christian, Holger, Jacek Generowicz, Ludovic Aubry, Olivier, Samuele, Laura) * Work on the annotator: - SomeBuiltin.call - implemented annotation for some built-ins that were missing - SomeSlice support and annotating some ops involving them - iter built-in - dict methods support for SomeDict - preserve instance types when unifying with None - dict object dummy value is a W_Root subclass instance, so as not to confuse the annotator - taught the annotator how to deduce the type of e in try: ... except SomeException, e: ... <-- here - removal of most *args call sites used in the multi-method slicing mechanism, like what was done for gateways. The annotator no longer chokes on such call sites. :-) - wrote a small test case to show the remaining problems with the current setup of BuiltinCode and BuiltinFrame subclasses and the highly polymorphic BuiltinCode.func callable attribute. (Adrien, Michael, Samuele, some design discussion with Armin) * Added shortcuts is_w(.,.) (= space.is_true(space.is_(.,.))), eq_w(.,.) (= space.is_true(space.eq(.,.))) to base object space. (Adrien) * Allow arguments to be named in the stdobjspace's type's methods. As a first example, this enables list.sort(cmp=...). (Armin) * Generated app-level exceptions module and from there using geninterplevel an interp-level module (in particular this circumvent bootstrap issues). (Christian) This implementation approach for built-in modules should be polished and generalized. * Added python library as of 2.3.4 to our tree as dist/lib-python-2.3.4. After discussion, it was agreed that this should be left unchanged and modules needing modifications should be copied to pypy/lib (renamed from appspace) and modified there, also new pypy-specific app-level modules should live there. That means that as general rule we should copy and borrow from this specific Python version, in the not too distant future we will switch to Python 2.4. Pypy specific modified or new app-level tests should go to pypy/lib/test2. (Holger) Work is needed on how those should be run, we would like to be able to run them both on PyPy and on CPython. * py.test support for unittest and output-comparing tests has been added to dist/lib-python-2.3.4 (this is a special exception to the above rules), as conftest.py and pypy_unittest.py in dist/lib-python-2.3.4/test. Look at conftest.py for details, there are lists of passing tests there (Anders, Holger) * Work on our sio based file implementation. (Armin did a major rework) * Support for __del__ on user-defined classes. For now borrowing functionality from underlying CPython. (Armin) * Work has been started on an app-level implementation for old-style classes, we have a good chunk of the implementation but it is not that much tested nor integrated for now. (Christian, Samuele) * Marcus Denker gave a presentation on Squeak, its continuations based web framework Seaside and Croquet. We discussed whether PyPy is a sensible approach to give in the future comprehensive Python support as scripting language to Squeak/Croquet. * We set tasks for the next few weeks, and now begin to prepare for the next Sprint, before PyCon. regards. Samuele and Laura PS: I (Samuele) did the listing of tech changes, sorry if I missed someone's credits.

2 1

Re: PyPy pure python crypt module
by holger krekel 02 Feb '05

02 Feb '05

Hello Jan, On Tue, Feb 01, 2005 at 23:12 +0100, Jan Dries wrote: > Hello, > > I noticed on http://www.python.org/moin/PyPySprint that you are looking > for a pure Python implementation of a number of modules, including the > crypt module. Note that for the crypt module there is such an > implementation available already: > > http://home.clear.net.nz/pages/c.evans/sw/ thanks for pointing that out! I have CCed pypy-dev with this information. The above crypt implementor notes that the code has some problem with int/longs which sounds familiar with PyPy :-) cheers, holger

1 0