Mailman 3 July 2006 - pypy-dev

Summer of PyPy: call for proposals
by hpk＠trillke.net July 21, 2006

July 21, 2006

Hi to all students! Happily, we are able to offer students mentoring and full sprint participant's funding if we receive a proposal outlining an interesting project related to PyPy and its development tools. This follows up on the "Summer of Code" campaign from Google but is completely independent from it and also works differently (i.e. there is no success bounty). See the full call for details: http://codespeak.net/pypy/dist/pypy/doc/summer-of-pypy.html and here is a list of … [View More]

1 0

Re: [pypy-dev] [pypy-svn] r30034 - pypy/dist/pypy/objspace/std
by Ben.Young＠risk.sungard.com July 21, 2006

July 21, 2006

Hi Xoraxax (sorry, I don't know your real name!), This looks really good! It shows there are still some major (easy) performance wins in PyPy. One question, why would you need any calls to str2object in any of the non-mutating methods? Surely the class knows that no non-strings have been added, and if __getitem__ is called with a non-string, you could just throw KeyError. I guess this is becuase you have to replicate the fact that hash is always called, but couldn't you do: … [View More]if space.is_w(w_lookup_type, space.w_str): return w_dict.content_str[space.str_w(w_lookup)] else: w_lookup.__hash__() raise KeyError ? Keep up the good work (to all PyPy's). Things are really starting to look good! Cheers, Ben pypy-svn-bounces(a)codespeak.net wrote on 14/07/2006 11:51:55: > Author: xoraxax > Date: Fri Jul 14 12:51:55 2006 > New Revision: 30034 > > Modified: > pypy/dist/pypy/objspace/std/dictstrobject.py > Log: > Speeded up dict creation, added shortcut for look ups of vars with > sane hashes, fixed get mm, fixed mutation handling in the iterators. > (manual merge of changes done in the other branch) > > Modified: pypy/dist/pypy/objspace/std/dictstrobject.py > ============================================================================== > --- pypy/dist/pypy/objspace/std/dictstrobject.py (original) > +++ pypy/dist/pypy/objspace/std/dictstrobject.py Fri Jul 14 12:51:55 2006 > @@ -1,4 +1,3 @@ > - > from pypy.objspace.std.objspace import * > from pypy.interpreter import gateway > > @@ -7,52 +6,52 @@ > class W_DictStrObject(W_Object): > from pypy.objspace.std.dicttype import dict_typedef as typedef > > - def __init__(w_self, space, w_otherdict=None): > + def __init__(w_self, space): > w_self.space = space > - if w_otherdict is None: > - w_self.content = None > - w_self.content_str = {} > - else: > - if w_otherdict.content is None: > - w_self.content = None > - w_self.content_str = w_otherdict.content_str.copy() > - else: > - w_self.content = w_otherdict.content.copy() > - w_self.content_str = None > + w_self.content = None > + w_self.content_str = {} > > def initialize_content(w_self, list_pairs_w): # YYY > for w_k, w_v in list_pairs_w: > w_self.setitem(w_k, w_v) > > def getitem(w_dict, w_lookup): > + """ Helper function, raises rpython exceptions. """ > space = w_dict.space > if w_dict.content is None: > - if space.is_w(space.type(w_lookup), space.w_str): > + w_lookup_type = space.type(w_lookup) > + if space.is_w(w_lookup_type, space.w_str): > return w_dict.content_str[space.str_w(w_lookup)] > else: > + if w_dict.isSaneHash(w_lookup_type): > + raise KeyError > w_dict.str2object() > return w_dict.content[w_lookup] > > - def get(w_dict, w_lookup): > + def get(w_dict, w_lookup, w_default): > + """ Helper function, raises rpython exceptions. """ > space = w_dict.space > if w_dict.content is None: > - if space.is_w(space.type(w_lookup), space.w_str): > - return w_dict.content_str.get(space.str_w(w_lookup), None) > + w_lookup_type = space.type(w_lookup) > + if space.is_w(w_lookup_type, space.w_str): > + return w_dict.content_str.get(space. > str_w(w_lookup), w_default) > else: > + if w_dict.isSaneHash(w_lookup_type): > + return w_default > w_dict.str2object() > - return w_dict.content.get(w_lookup, None) > + return w_dict.content.get(w_lookup, w_default) > > > def setitem(w_self, w_k, w_v): > space = w_self.space > - if w_self.content is None: > + if w_self.content is not None: > + w_self.content[w_k] = w_v > + else: > if space.is_w(space.type(w_k), space.w_str): > w_self.content_str[space.str_w(w_k)] = w_v > else: > w_self.str2object() > w_self.content[w_k] = w_v > - else: > - w_self.content[w_k] = w_v > > set_str_keyed_item = setitem > > @@ -65,7 +64,17 @@ > for k, w_v in w_self.content_str.items(): > w_self.content[w_self.space.wrap(k)] = w_v > w_self.content_str = None > - > + > + def isSaneHash(w_self, w_lookup_type): > + """ Handles the case of a non string key lookup. > + Types that have a sane hash/eq function should allow us to > return True > + directly to signal that the key is not in the dict in any case. > + XXX The types should provide such a flag. """ > + > + space = w_self.space > + # XXX there are much more types > + return space.is_w(w_lookup_type, space.w_NoneType) or > space.is_w(w_lookup_type, space.w_int) > + > def __repr__(w_self): > """ representation for debugging purposes """ > return "%s(%s,%s)" % (w_self.__class__.__name__, w_self. > content, w_self.content_str) > @@ -144,7 +153,11 @@ > > def contains__DictStr_ANY(space, w_dict, w_lookup): > if w_dict.content is None: > - if not space.is_w(space.type(w_lookup), space.w_str): > + w_lookup_type = space.type(w_lookup) > + if not space.is_w(w_lookup_type, space.w_str): > + if w_dict.isSaneHash(w_lookup_type): > + return space.w_False > + #foo("degenerated in contains: " + space. > type(w_lookup).getname(space, '?')) > w_dict.str2object() > else: > return space.newbool(space.str_w(w_lookup) in w_dict.content_str) > @@ -247,7 +260,14 @@ > return w_res > > def dict_copy__DictStr(space, w_self): > - return W_DictStrObject(space, w_self) > + w_new_dict = W_DictStrObject(space) > + if w_self.content is None: > + w_new_dict.content = None > + w_new_dict.content_str = w_self.content_str.copy() > + else: > + w_new_dict.content = w_self.content.copy() > + w_new_dict.content_str = None > + return w_new_dict > > def dict_items__DictStr(space, w_self): > if w_self.content is not None: > @@ -291,9 +311,7 @@ > w_self.content.clear() > > def dict_get__DictStr_ANY_ANY(space, w_dict, w_lookup, w_default): > - if w_dict.content is None: > - return w_dict.content_str.get(space.str_w(w_lookup), w_default) > - return w_dict.content.get(w_lookup, w_default) > + return w_dict.get(w_lookup, w_default) > > app = gateway.applevel(''' > def dictrepr(currently_in_repr, d): > @@ -348,6 +366,12 @@ > def return_entry(w_self, w_key, w_value): > raise NotImplementedError > > + def handleMutation(w_self): > + space = w_self.space > + w_self.len = -1 # Make this error state sticky > + raise OperationError(space.w_RuntimeError, > + space.wrap("dictionary changed size during iteration")) > + > registerimplementation(W_DictStrIterObject) > > class W_DictStrIter_Keys(W_DictStrIterObject): > @@ -435,16 +459,25 @@ > return w_dictiter > > def next__DictStrIterObject(space, w_dictiter): > - # XXX hum, this is broken currently > - if w_dictiter.content_str is not None and w_dictiter. > w_dictobject.content_str is None: > - raise OperationError(space.w_RuntimeError, > - space.wrap("you stupid dictionary user - you have > just done an operation internally mutating the dictionary")) > - > - if w_dictiter.content is not None: > + # iterate over the string dict even if the dictobject's data was forced > + # to degenerate. just bail out if the obj's dictionary size differs. > + if (w_dictiter.content_str is not None and w_dictiter. > w_dictobject.content_str is None > + and len(w_dictiter.w_dictobject.content) != w_dictiter.len): > + w_dictiter.handleMutation() > + > + if w_dictiter.content_str is not None: > + if w_dictiter.len != len(w_dictiter.content_str): > + w_dictiter.handleMutation() > + # look for the next entry > + w_result = w_dictiter.next_entry() > + if w_result is not None: > + w_dictiter.pos += 1 > + return w_result > + # no more entries > + w_dictiter.content_str = None > + elif w_dictiter.content is not None: > if w_dictiter.len != len(w_dictiter.content): > - w_dictiter.len = -1 # Make this error state sticky > - raise OperationError(space.w_RuntimeError, > - space.wrap("dictionary changed size during iteration")) > + w_dictiter.handleMutation() > # look for the next entry > w_result = w_dictiter.next_entry() > if w_result is not None: > @@ -452,19 +485,6 @@ > return w_result > # no more entries > w_dictiter.content = None > - else: > - if w_dictiter.content_str is not None: > - if w_dictiter.len != len(w_dictiter.content_str): > - w_dictiter.len = -1 # Make this error state sticky > - raise OperationError(space.w_RuntimeError, > - space.wrap("dictionary changed size during > iteration")) > - # look for the next entry > - w_result = w_dictiter.next_entry() > - if w_result is not None: > - w_dictiter.pos += 1 > - return w_result > - # no more entries > - w_dictiter.content_str = None > raise OperationError(space.w_StopIteration, space.w_None) > > def len__DictStrIterObject(space, w_dictiter): > _______________________________________________ > pypy-svn mailing list > pypy-svn(a)codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn > [View Less]

2 4

Tomorrow morning...
by Antonio Cuni July 20, 2006

July 20, 2006

... I will graduate!! :-) :-) :-) A big, big 'thank you' to all pypy developers, for having welcomed and supported me during last months: it has been very nice to work with you (and I hope to continue in future). I still remember my first post on pypy-dev: I was "Looking for a thesis" (cit.) and I was rather confused about pypy architecture: it was only 5 months ago, but it seems years. Then I started coding and I've never stopped :-). For those interested in, the final version of my … [View More]

5 4

return of the return of pypy-sync
by Michael Hudson July 20, 2006

July 20, 2006

Once again, I'd like to invite all active PyPy developers to a short (no more than half an hour) sychronisation meeting in #pypy-sync on freenode at 17:30 UTC+2 today. Agenda: - status pre-prepared 3-line reports of what you did last week, what you plan to do next week and what is blocking your progress at this time. this is the area i see as the most important this week: work has begun to spread in many directions, and it's a good idea to have an idea who is doing what … [View More]

1 0

Implenting -m command line option
by Leonardo Soto July 19, 2006

July 19, 2006

Hi all, I'm new to pypy, and to get myself started I'm trying to fix #145, that is, implement the -m flag. So I did search on the web and python sources looking for what exactly -m means, and PEP-338[1] has the better explanation: "If the module is found, and is of type PY_SOURCE or PY_COMPILED, then the command line is effectively reinterpreted from python <options> -m <module> <args> to python <options> <filename> <args>. This includes setting sys.argv[0]… [View More]

2 1

Security ideas
by Armin Rigo July 18, 2006

July 18, 2006

Hi all, On Monday I was at an inspiring seminar about (a specific form of) language-level security. I've collected the PyPy-ification of these ideas there: http://codespeak.net/svn/pypy/dist/pypy/doc/discussion/security-ideas.txt Although the focus is different, it makes me think that we could also use similar ideas to implement a form of 'rexec' (restricted execution), with functions compiled by secure() as in the draft above, but running at a priviledge level which is lower than the … [View More]

6 7

cpy object space
by Jiwon Seo July 15, 2006

July 15, 2006

pypy/bin/py.py does not work with cpy object space [llcoolj] bin > ./py.py -o cpy [cbuild:execute] cc -O2 -pthread -I/usr/include/python2.4 -c ctypesplatcheck_0.c -o ctypesplatcheck_0.o [cbuild:execute] cc -pthread /tmp/usession-23/ctypesplatcheck_0.o -lm -lpthread -o /tmp/usession-23/ctypesplatcheck_0 Traceback (most recent call last): File "./py.py", line 204, in ? sys.exit(main_(sys.argv)) File "./py.py", line 82, in main_ space.setitem(space.sys.w_dict,space.wrap('executable'… [View More]

1 1

fixing ireland sprint (21th-27th august)
by hpk＠trillke.net July 14, 2006

July 14, 2006

Hi folks, i took over trying to organise the Ireland sprint (scheduled 21st - 27th August currently) but i had problems getting feedback from the organisers yet. Therefore, i am not sure about some details but i started a draft announcement anyway: extradoc/sprintinfo/ireland-2006/announcement.txt and also listed people from whom i have heard that they want to come extradoc/sprintinfo/ireland-2006/announcement.txt Please check and add yourself already. However, how do we go … [View More]

1 0

PyPy is faster than CPython (given an evil enough benchmark)
by Carl Friedrich Bolz July 12, 2006

July 12, 2006

Hi all! Together with the optimized string objects that Armin, Pieter, Alexander and me have been working on, it is possible to construct a (very corner-casish) benchmark, which exploits the fast cases of PyPy in such a way, that it is quite a bit faster CPython. See the following code: import time, sys def test_speed(num): t1 = time.time() result = "" intermediate = [] for i in range(num): result += str(i) intermediate.append(result) t2 = time.time() … [View More]

3 2

Bug hunting in rctime becomes wtf
by Lawrence Oluyede July 11, 2006

July 11, 2006

I had a nasty bug like this one in the compilation process of the rctime module: [translation:ERROR] Error: [translation:ERROR] TyperError: non-constant tuple index [translation:ERROR] .. block@34 with 1 exits [translation:ERROR] .. v0 = getitem(tup_0, ___i_0) You can reproduce it decommenting asctime function from interp_time.py and in the __init__.py file of the rctime module. After some digging I've found the source of the problem in the code Armin and I have checked-in at the sprint to … [View More]

1 0