Mailman 3 February 2006 - pypy-dev

pypy-sync developer meeting 16th February 2006
by Michael Hudson 16 Feb '06

16 Feb '06

Time & location: 1pm (30 minutes), GMT+1 at #pypy-sync Attendees:: Carl Friedrich Bolz Anders Chrigstrom Aurelien Campeas Nik Haldimann Holger Krekel Samuele Pedroni Eric van Riet Paap Armin Rigo Michael Hudson (moderation & minutes) Regular Topics ==================== - activity reports (3 prepared lines of info). All Attendees submitted activity reports (see the `IRC Log`_ and 'LAST/NEXT/BLOCKERS' entries in particular) - resolve conflicts/blockers: No blockers Topics of the week =================== - pycon sprint planning Michael agreed to update our sprint tutorial materials. There was consensus that the "Paris style" of daily planning meetings and small-ish discussion groups presenting results was likely to be a good approach, although flexibility is required as we don't really know who's going to be there. Arre agreed to see if the Strakt projector could be brought along. - mozart/oz sprint Shortly before the meeting Nicolas checked in (revision 23397, pypy/extradoc/sprintinfo/louvain-la-neuve-2006) enough information to answer all questions :) - extra core sprint There was agreement that we need a core sprint before EuroPython. Armin suggested the middle or end of March in Leysin, but most other people preferred the beginning of April. Armin agreed to investigate a sprint location in Leysin for the dates 3-9th of April. So if you didn't attend the meeting, do want to attend the sprint and can't make these dates, now would be the time to shout :) - state of stackless The WP9 work requires an efficient threading implementation. Aurelien wanted to know if the stackless work had progressed to the point of making an efficient green threads implementation to be made. Armin said that anything was probably possible, but that precise details of what was needed were required. There was agreement that there should be a session on this at a sprint, or if that failed a planned session on IRC about the topic. The issue not especially time-critical, fortunately. - moderation of pypy-sync meetings There was consensus that too many people will be in the air to have a pypy-sync meeting next week. As most of those people will still be at pycon a week later for the next pypy-sync meeting, it's not clear that there will be one on the 2nd of March either. .. _`IRC log`: Complete IRC log ================= :: [12:55] <mwh> ok, my clock says 12:02 [12:55] arigo (n=arigo(a)p54AB84C0.dip0.t-ipconnect.de) joined #pypy-sync. [12:55] <mwh> so i'd like to start [12:55] <mwh> activity reports, please [12:55] <auc> LAST: finished streams, scheduling problem [12:55] <auc> NEXT: exercise the solver with various problems, dbugging [12:55] <auc> BLOCK: nil [12:56] <mwh> LAST: genc work [12:56] <mwh> NEXT: more genc, pycon preparations [12:56] <mwh> BLOCKERS: - [12:56] <nikh> LAST: random explorations and fixes [12:56] <nikh> NEXT: more explorations, also around rctypes and ootypesystem [12:56] <nikh> BLOCKERS: none [12:56] <ericvrp> LAST: stackless optimization and transformation, NEXT: more of that, BLOCKERS: - [12:56] <hpk> LAST: codespeak refinements, press release, non-pypy [12:56] <hpk> NEXT: pycon + sprints, customer-meeting [12:56] <hpk> BLOCKERS: [12:56] <arre> PREV: JIT-work [12:56] <arre> NEXT: JIT-work [12:56] <arre> BLOCKERS: None [12:56] <cfbolz> LAST: some GC work with Armin: finding the roots sort of works somewhat [12:56] <cfbolz> NEXT: maybe more GC work [12:56] <cfbolz> BLOCKERS: not enough time [12:57] <mwh> arigo, pedronis ? [12:57] <pedronis> LAST: jit work, help with GC, some opt and polish, bug fixingwork [12:57] <pedronis> NEXT: jit,, pycon [12:57] <pedronis> BLOCKERS: - [12:57] <arigo> LAST: Psyco stuff, GC framework progress with Carl [12:57] <arigo> NEXT: more JIT, possibly more GC [12:57] <arigo> BLOCKERS: - [12:57] <mwh> ok [12:57] <mwh> the first topic is pycon sprint planning [12:58] <mwh> i'm not sure what we need to do other than update our tutorial [12:58] <mwh> (also, we have some talks to write, i think...) [12:58] Action: hpk will take care for the py lib/py.test track of that sprint [12:58] <hpk> i guess we are going to proceed similarly to paris [12:58] <mwh> does anyone desperately want to work on the tutorial? [12:59] <mwh> hpk: in which way do you mean? [12:59] <hpk> mwh: having short "result presesntation and rough planning" daily sessions and otherwise work more in sub-groups [12:59] <mwh> hpk: ah right [12:59] <hpk> arre,pedronis: can we have the projector there? [12:59] <mwh> i think we probably need to be flexbile (agile, even :) and see who turns up [13:00] <hpk> yes [13:00] <hpk> arigo: can you also prepare our architecture session talk a bit before the conference? [13:00] <mwh> if noone else wants to work on the tutorial, i can do it [13:00] <arre> hpk: Provided Jacob does not want it I'll make sure to bring it allong. [13:00] <hpk> mwh: would be great - i can review it at the conference but not before i guess [13:01] <mwh> we've got a fair few topics to get though today, don't want to talk about this for two long [13:01] <arigo> hpk: yes [13:01] <mwh> so, decisions: arre to check about beamer, me to look at tutorial [13:01] <mwh> any dissent? [13:01] <hpk> no, fine [13:02] <mwh> - mozart/oz sprint [13:02] <mwh> i think for this we'd just like to hear dates, plans, who's going [13:02] <auc> nicolas just posted some bits of information about that [13:02] <mwh> i take it this is an insiders only sprint [13:02] <auc> into svn [13:02] <mwh> auc: ah [13:02] <cfbolz> mwh: the sprint announcement does not sound like it [13:02] <hpk> well, it can't hurt to do an announcement at least to pypy-dev? [13:02] <mwh> cfbolz: ok [13:03] <mwh> ah yes, i see the check in now [13:03] <auc> http://codespeak.net/svn/pypy/extradoc/sprintinfo/louvain-la-neuve-2006/ [13:03] <mwh> ok, well i guess that check in answers all the questions [13:03] <mwh> move on to next topic? [13:04] <cfbolz> mwh: yes :-) [13:04] <mwh> - extra core sprint [13:04] <mwh> i don't think there's much doubt that we need another sprint before europython [13:05] <mwh> as both pycon and tokyo will be dissemination focussed to some extent [13:05] <mwh> and louvain-la-neuve is a bit off centre [13:05] <cfbolz> mwh: agreed [13:05] <arigo> there is a non-concrete idea about Leysin, but more mid-end-of-March than early April [13:05] <mwh> arigo: i see [13:05] <hpk> arigo: early april would not be possible? [13:05] <mwh> i can't make the end of march [13:06] Action: hpk will not make it mid/end march either [13:06] <arigo> the longer we wait, the less snow we have :-/ [13:06] <cfbolz> arigo: I would also prefer early april :-) [13:06] <mwh> arigo: hmm :) [13:06] <mwh> i don't know if we want to try and decide dates now [13:06] <mwh> just "yes, we need a sprint" [13:07] <hpk> well, it seems that early april is more likely to fit at least with carl, michael and me [13:07] <arigo> yes, we need a sprint [13:07] <hpk> just time-wise [13:07] <cfbolz> mwh: yes, we need a sprint [13:07] <arigo> hpk: point taken [13:07] <pedronis> mid of march doesn't work for me either [13:07] <mwh> auc: how about logilab? [13:07] <auc> don't know [13:07] <mwh> it seems getting logilab and tismerysoft in the same place at some point would be a good thing [13:08] <mwh> (see next topic...) [13:08] <cfbolz> indeed [13:09] <auc> but april is better than march [13:09] <hpk> ericvrp: how is beginning april for you? [13:09] <ericvrp> I don't think I can make it to an intermediatte sprint [13:09] <mwh> arigo: is leysin-in-april still a workable plan? [13:09] <hpk> ericvrp: ok, i thought that because you don't come to pycon ... but then you'd like to go to tokyo, i see [13:09] <mwh> (apart from the snow issue) [13:10] <arigo> mwh: I propose that we fix potential dates (to be checked by me): [13:10] <ericvrp> maybe I can come for a very short while, if that makes sense at all [13:10] <arigo> the week of the 3rd-7th of April [13:10] <hpk> arigo: yes, sounds like it [13:10] <mwh> arigo: yes [13:10] <hpk> and people can come around (preferably before) if they want to ski [13:10] <arigo> if we don't fix dates now it will slip later Yet Another Time :-) [13:11] <mwh> (predictably enough i'm prancing around in rock shoes the previous week...) [13:11] <hpk> arigo: will you check out the old setting? [13:11] <mwh> arigo: yes, agreed [13:11] <mwh> 3-7? [13:11] <arigo> (when's Easter?) [13:11] <arigo> hpk: the same place, yes [13:11] <mwh> easter sunday is 23rd april [13:11] <mwh> i think [13:11] <arre> arigo: The week after. [13:11] <hpk> 3-7th sounds a bit like no-breakday? [13:11] <hpk> it's five days [13:12] <arigo> arre: 16th? thanks [13:12] <arigo> hpk: I talked about the week [13:12] <hpk> ah ok [13:12] <hpk> so we are talking about 3-9th with a breakday in between or so [13:12] <mwh> er, yes 16th [13:12] <cfbolz> yes, sounds good to me [13:12] <mwh> 3-9 works for me [13:13] <hpk> pedronis, arre: would work for you as well? [13:13] <mwh> so, mention this on pypy-dev? [13:13] <arre> 3-9 is not a problem for me. [13:13] <pedronis> it works for me [13:13] <arigo> mwh: ok (I need to check of course) [13:13] <mwh> i'd like to move on to the next topic soon ish [13:13] <mwh> arigo: of course [13:13] <hpk> arigo: yip [13:13] <hpk> fine with me, mwh: i think it's enough for now to mention it prominently in the minutes [13:14] <mwh> hpk: ok [13:14] <mwh> next topic: [13:14] <mwh> - state of stackless [13:14] Action: ericvrp Christian needs to answer that one [13:14] <mwh> this is auc's topic, but basically i think the question is particularly about whether there are concrete plans for microthreads [13:15] <mwh> ericvrp: yeah [13:15] <mwh> it's about 2am where he is though, i think [13:15] <arigo> I think Christian's got coroutines, greenlets and tasklets so far [13:15] <auc> what are green/tasklets ? [13:15] <auc> wrt coroutines ? [13:15] <arigo> and it should be very easy to add other abstractions [13:15] <mwh> arigo: is that enough to build microthreads at app level? [13:15] <arigo> auc: mostly just a different, slightly richer interface than bare coroutines [13:16] <mwh> i guess not with preemption [13:16] <auc> mwh: sure [13:16] <arigo> mwh: yes, it all depends on what is needed more precisely [13:16] <cfbolz> indeed [13:16] <auc> anyway real threads are needed, if only because they are part of python, no ? [13:16] <mwh> i guess pypy-sync isn't really the forum for this discussion in the long run [13:16] <cfbolz> auc: well, yes. but you cannot have thousands of them [13:17] <auc> sure, that's why we want picrothreads [13:17] <auc> (shit) [13:17] <auc> it's not extremely urgent [13:17] <cfbolz> picothreads? [13:17] <arigo> auc: the answer is that it's mostly easy for any of us to build whatever you need, but it must be more well-defined [13:17] <auc> micro, nano, pico, femto, whatever ;-) [13:18] <mwh> auc: good to know (about the urgency) [13:18] <cfbolz> auc: well, would you want the exactly the same interface as threads, for example [13:18] <auc> arigo: it's simple, we want efficient threads [13:18] <mwh> ideal would be to have a session on this at a sprint [13:18] <auc> bonus : threads that return values [13:18] <arigo> auc: but for example, do you need threads that can block on system calls without blocking other threads [13:18] <auc> bonus : threads that expose the interface present in Oz [13:18] <arigo> auc: I guess not [13:19] <auc> arigo: I guess yes [13:19] <mwh> if not, would it make sense to schedule some irc session to get the relevant parties together? [13:19] <arigo> auc: that's very hard to do without real OS threads [13:19] <auc> this interface is describe in some document I've committed already [13:19] <arigo> mwh: yes [13:19] <auc> I can move it to doc/discussions [13:19] <cfbolz> yes please [13:19] <auc> arigo: ok we'll see [13:20] <mwh> so ideally this will be talked about at lln/leysin [13:20] <mwh> otherwise irc sometime [13:20] <mwh> now for two minutes: [13:20] <mwh> who moderates the next pypy-sync? [13:20] <mwh> it's not going to be me! [13:20] <arigo> (auc: feel free to talk about in in #pypy at any time :-) [13:21] <cfbolz> mwh: will there be enough people on next thursday? [13:21] <nikh> aren't about half of the people present here going to be on planes next pypy-sync? [13:21] <arigo> 23rd? half of us are in the plane [13:21] <mwh> cfbolz: don't know [13:21] <hpk> nikh: true [13:21] <hpk> don't planes have wireless these days? [13:21] <mwh> ok, so maybe there will not be a pypy-sync next week [13:21] <mwh> that's fine [13:22] <mwh> hpk: only if you pay $$$$ i think [13:22] <auc> arigo: yup [13:22] <cfbolz> mwh: then let's plan the one afterwards [13:22] <mwh> well, the one afterwards is on the day i'm travelling again [13:22] <cfbolz> argh [13:22] <mwh> it will be about 5am local time for pycon-ers, i think [13:23] <hpk> hum [13:23] <hpk> well, then let's not decide now in a hurry [13:23] <mwh> ok [13:23] <mwh> there are maybe 30 seconds for any final points :) [13:24] <hpk> (we could take the opportunity to shift it to a different time) [13:24] <mwh> consistency is good though [13:24] <mwh> anyway, time up [13:24] Action: mwh closes the meeting -- 7. It is easier to write an incorrect program than understand a correct one. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

1 0

web site improvements
by Nicolas Chauvat 16 Feb '06

16 Feb '06

Hi, I am willing to spend some hours improving the website. I understood that commiting changes to pypy/doc could be enough, but where is the top menu defined ? Changes I would like to make: - visible "events" info with sprints and conferences. - smaller font by default (css change would do) - visible link to SVN - shorter "getting started" getting to more info - simpler and more visible "here is what pypy is" section or page - probably others that would come to mind... -- Nicolas Chauvat logilab.fr - services en informatique avancée et gestion de connaissances

2 1

#pypy-sync tomorrow!
by Michael Hudson 16 Feb '06

16 Feb '06

Good afternoon! It is my pleasure to invite all active pypy developers to the weekly pypy-sync meeting which will take place tomorrow (the 16th of February) at 1pm CET (12 noon GMT) on the #pypy-sync channel on the freenode IRC network. The topics known at this point are: - three line activity reports as usual. - pycon sprint planning we should at the very least make sure our tutorial material is up to date before the pycon sprints. who will do this? - mozart/oz sprint some of the developers who are not going to this sprint are curious as to what the goals of this sprint are. a brief summary of what it's about would be appreciated. - extra core sprint we are probably going to want to have another core sprint before EuroPython, as both the PyCon and Tokyo sprints are likely to be rather beginner/dissemination oriented. where and when? i've heard that early april in leysin is a possibility. If there are any other topics you would like to discuss, please let me know by email. Cheers, mwh -- My hat is lined with tinfoil for protection in the unlikely event that the droid gets his PowerPoint presentation working. -- Alan W. Frame, alt.sysadmin.recovery

2 2

coroutine problem
by Christian Tismer 14 Feb '06

14 Feb '06

Hi Armin, here a summary of my problem. This version compiles: def _bind(self, thunk): state = self.costate self.parent = state.current state.last.frame = yield_current_frame_to_caller() try: thunk.call() except CoroutineExit: # ignore a shutdown exception pass except Exception, e: # redirect all unhandled exceptions to the parent state.things_to_do = True state.temp_exc = e while self.parent is not None and self.parent.frame is None: # greenlet behavior is fine self.parent = self.parent.parent return self._update_state(state, self.parent) WHen I change it like this: def _bind(self, thunk): state = self.costate self.parent = state.current try: state.last.frame = yield_current_frame_to_caller() thunk.call() Then I get the exception mismatch. I still don't get why you think these are identical. It is true, the yield call does never create an exception by itself. But it is the place where I arrive from any other switch, and the switch is built in a way that it can create an exception. This pops up exactly at the place where we yield. So I'd like to show up inside an exception handler. Of course it is possible to move this handler further up. anything wrong with this reasoning? cheers - chris -- Christian Tismer :^) <mailto:tismer@stackless.com> tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/

2 2

Pointer To Member Problem with rctypes
by Gerald Klix 14 Feb '06

14 Feb '06

Hi all, I have a problem with GcStruct, Struct and Ptr I can not resolve with my own devices. After some trial with ctypes running on CPython I came up with the following lowlevel representation for structures: Ptr( GcStruct( "CtypesGcStructure_<ClassName> ( "c_data" (Struct "C-Data_<ClassName> *<Fieldefintions>) ) ) ) Samuele and Armin helped to implement rtype_getattr and rtype_setattr that delegate all attribute access to the c_data field, by pointing out how to implement a helper method called generateCDataAccess: def generateCDataAccess( self, variable, lowLevelOperations ): """ Answer the C level data sub structure. """ inputargs = [ variable, inputconst( Void, "c_data" ) ] return lowLevelOperations.genop( "getfield", inputargs, # Using Ptr makes the C-compiler barf, # omitting Ptr leeds to specialisation errors Ptr( self.c_data_lowleveltype ) ) def rtype_setattr( self, highLevelOperation ): c_data = self.generateCDataAccess( highLevelOperation.inputarg( self, 0 ), highLevelOperation.llops ) inputargs = highLevelOperation.inputargs( *highLevelOperation.args_r[ :3 ] ) inputargs[ 0 ] = c_data print "inputargs:", inputargs print "r_result:", highLevelOperation.r_result highLevelOperation.genop( "setfield", inputargs ) Obviously generateCDataAccess needs to return a Ptr to the embedded C-structure. When I use Ptr as in the example above, I get the following traceback: c:\Dokumente und Einstellungen\root\Lokale Einstellungen\Temp\usession-353\testi ng_2\testing_2.c(894) : error C2440: '=': 'pypy_C_Data_tagpoint0' kann nicht in 'pypy_C_Data_tagpoint0 *' konvertiert werden Which means, that a 'pypy_C_Data_tagpoint0' can not be converted to a 'pypy_C_Data_tagpoint0 *'. A brief look at the generated C-source, shows that the compiler is right here: struct pypy_CtypesStructure_tagpoint0 *pypy_g__py_test_compile_struct(struct pypy_CtypesStructure_tagpoint0 *l_p_0, long l_x_1, long l_y_1) { /*void l_v111;*/ /*void l_v112;*/ struct pypy_C_Data_tagpoint0 *l_v98; struct pypy_C_Data_tagpoint0 *l_v99; struct pypy_CtypesStructure_tagpoint0 *l_v110; pypy_IncRf_CtypesStructure_tagpoint0(l_p_0); block0: l_v98 = l_p_0->cs_c_data; l_v98->cd_x = l_x_1; l_v99 = l_p_0->cs_c_data; l_v99->cd_y = l_y_1; l_v110 = l_p_0; goto block1; block1: return l_v110; } If I ommit the Ptr construction so that, generateCDataAccess is defined as follows: def generateCDataAccess( self, variable, lowLevelOperations ): """ Answer the C level data sub structure. """ inputargs = [ variable, inputconst( Void, "c_data" ) ] return lowLevelOperations.genop( "getfield", inputargs, self.c_data_lowleveltype ) I get the following error: [D:\svnwork\work\dist\pypy\rpython\lltypesystem\lltype.py:146] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ def _nofield(self, name): raise AttributeError, 'struct %s has no field %r' % (self._name, E name) > AttributeError: struct C-Data_tagpoint has no field 'TO' This is no surprise at all. The getfield operation obviosly needs a pointer to work with. So it boils down to the question: How can I access the members of an embedded structure? Is there a possibility to get the address of substructure, just like C's ampersand (&) operator does? Any help is welcome. Best regards, Gerald

2 1

PyCon 2006 sprint
by Armin Rigo 11 Feb '06

11 Feb '06

Post-PyCon PyPy Sprint: February 27th - March 2nd 2006 ============================================================ The next PyPy sprint is scheduled to take place right after PyCon 2006 in Dallas, Texas, USA. We hope to see lots of newcomers at this sprint, so we'll give friendly introductions. Note that during the Pycon conference we are giving PyPy talks which serve well as preparation. Goals and topics of the sprint ------------------------------ While attendees of the sprint are of course welcome to work on what they wish, we offer these ideas: - Work on an 'rctypes' module aiming at letting us use a ctypes implementation of an extension module from the compiled pypy-c. - Writing ctypes implementations of modules to be used by the above tool. - Experimenting with different garbage collection strategies. - Implementing Python 2.5 features in PyPy - Implementation of constraints solvers and integration of dataflow variables to PyPy. - Implement new features and improve the 'py' lib and py.test which are heavily used by PyPy (doctests/test selection/...). - Generally experiment with PyPy -- for example, play with transparent distribution of objects or coroutines and stackless features at application level. - Have fun! Location -------- The sprint will be held wherever the PyCon sprints end up being held, which is to say somewhere within the Dallas/Addison Marriott Quorum hotel. For more information see the PyCon 06 sprint pages: - http://us.pycon.org/TX2006/Sprinting - http://wiki.python.org/moin/PyCon2006/Sprints Exact times ----------- The PyPy sprint will from from Monday February 27th until Thursday March 2nd 2006. Hours will be from 10:00 until people have had enough. Registration, etc. ------------------ If you know before the conference that you definitely want to attend our sprint, please subscribe to the `PyPy sprint mailing list`_, introduce yourself and post a note that you want to come. Feel free to ask any questions or make suggestions there! There is a separate `PyCon 06 people`_ page tracking who is already planning to come. If you have commit rights on codespeak then you can modify yourself a checkout of http://codespeak.net/svn/pypy/extradoc/sprintinfo/pycon06/people.txt .. _`PyPy sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint .. _`PyCon 06 people`: http://codespeak.net/pypy/extradoc/sprintinfo/pycon06/people.txt

3 6

problems compiling with complex [2]
by Stephan Diehl 11 Feb '06

11 Feb '06

sorry, forgot to mention that pypy/objspace/std/model.py contains a flag WITHCOMPLEX which must be set to 'True' in order to use the new complex implementation. Stephan

1 0

Next pypy-sync meeting
by Armin Rigo 01 Feb '06

01 Feb '06

Hi all, I'm happy to announce tomorrow's pypy-sync meeting, as usual, at 1pm on #pypy-sync on irc.freenode.net. Topics: * everybody's three-liners (last, next, blockers) * finalize the PyCon sprint announcement -- in particular, the sprint topics * py.test: Holger would like to ask us what we think should be better for running PyPy tests. For example, I think that py.test usually dumps far too much information with PyPy, and it's tedious to have to dig for the most important stuff -- e.g. the actual exception is usually lost between a long traceback and a long stdout dump. Please come up with complains and optionally with ideas on how to fix them :-) A bientot, Armin

1 0