Hi, this sounds interesting ;-) I was wondering how you were planning on identifying concrete tasks and assigning and tracking them, and keeping status visible to support cooperation. Do you already have CVS and bug/issue tracking methodology settled? I.e., such stuff is nuisance overhead or sanity preservation depending on the task, and the development phase, and scale/distribution of effort. Is there one single home page that leads to everything relevant, with some indication of historical vs current vs fire alarm links? Googling pypy got me (in two steps) to http://twistedmatrix.com/users/jh.twistd/moin/moin.cgi/PyPy Should that page be "claimed"? Here are some things that occur to me to think about in making a roadmap for your project. If you would make something like it to reflect your _actual_ project plans, it would help me (and maybe some other lurkers ;-) get an idea of what you are really up to ;-) 1.0 Historical background 1.1 Psyco 1.2 core group forms and evolves pypy idea 1.3 core group decides to go public, announces pypy-dev list 2.0 Initial pypy-dev discussions 2.1 Happening now ;-) 2.2 Preliminary expressions of interest, ideas 3.0 Straw-man roadmap 3.1 Decide hosting, bug/issue tracking methodology issues 3.1.1 Designation of primary info channels 3.1.1.1 Discussion - pypy-dev list, presumably 3.1.1.2 Current task/subproject/bugfix assignments/status 3.1.1.3 Single official page as comprehensive root of info tree 3.1.2 Sourceforge? 3.1.2.1 mirror/backup/file authentication issues? 3.1.3 pypy SDK? Recommended win32 & unix tool kits, compiler versions etc. 3.1.4 Windows vs unix issues 3.1.4.1 Maintain parallel MSVC++ project files like CPython's PCbuild directory? 3.1.4.2 Platform-independent builds? 3.2 Language issues 3.2.1 Decide on minimal core language bootstrap subset? 3.2.1.1 Identify needs for supporting 4.x below 3.2.1.2 Threading/locking/synchronizing issues? 3.2.2 Compiler hints/directives - what info & how? 3.2.2.1 command line opts? 3.2.2.2 config files? 3.2.2.3 source-embedded directives/pragmas/etc? 3.3.3.4 Preprocessing? 3.3 Other implementation issues? 3.3.1 Foreign function/C interface 3.3.2 New VM? Intermediate language representation? 3.3.3 Memory allocation/garbage collection/reference counting 3.3.4 Resource monitoring (guaranteed timely finalization?) 3.3.5 Security support 3.3.5.1 Sandboxing, restricted execution, resource quotas? 3.3.5.2 Special considerations for CGI/server contexts? 3.3.6 Checkpointable execution support, fastload images? 3.4 How to minimize platform C library dependencies for bootstrap core? 3.4.1 Strategy for weaning from temporary uses 3.5 Decide on initial build targets 3.5.1 Executables: Win32, unix, bootable images? 3.5.1 Libraries: DLLs, .so's 4.0 After the minimal core bootstrap language works 4.1 Re-implementation strategy using core boostrap language to write next level of language features. 4.1.1 How many levels of subset bootstrapping? Is it a hierarchy? 4.2 Special language requirements for writing what is now C in Cpython? Well, that's all the thoughts I have for now ;-) Regards, Bengt Richter
Off the top of my head, there's quite a bit of Python that has been reimplemented in Python available already: - Nearly all the import machinery (less the new PEP 302 stuff) is implemented in ihooks.py. - Command line reading is in code.py and codeop.py - A bytecode compiler is in the compiler package - A re-based tokenizer is in tokenize.py - I think that a Python implementation of marshal is (or was) part of Jython; I vaguely recall writing one when Jython was young The parser (needed by the bytecode compiler) is missing, but it's not particularly difficult to write. Not sure if this helps, but it's all I can contribute right now. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
Off the top of my head, there's quite a bit of Python that has been reimplemented in Python available already:
Yes, I know, and I appreciate it very much!
- Nearly all the import machinery (less the new PEP 302 stuff) is implemented in ihooks.py.
Absolutely. The amount of Python-coded stuff makes me so confident that this project is doable at all.
- Command line reading is in code.py and codeop.py
This is a pleasure, well known.
- A bytecode compiler is in the compiler package
Very good, I know. Finally, it is calling back into the C part, for some AST stuff and the parsetok. Not too har to implement, despite the fact that character-wise operation in Python isn't very effective at all. But there's sre.
- A re-based tokenizer is in tokenize.py
That's what I've overlooked. I thought we'd have to implement all of the parsetok stuff, since (as far as I analysed it) the compiler package calls back into the parser module, which still uses parsetok. This also touches one of my weak points: Is it good to replace C code by something which is depending on another special module like sre? This makes sre into something crucial to this re-implementation. But I'm not sure if this is a good way to go. I'm also not sure if it is good to re-implement certain modules using the common Python tricks and optimizations. To some extent, I have the impression that doing it the simple way, basically as done in C, would fit the optimizations of Psyco better. But that is an open question until we get some feedback from Armin Rigo.
- I think that a Python implementation of marshal is (or was) part of Jython; I vaguely recall writing one when Jython was young
Thanks a lot, I will try to look that up.
The parser (needed by the bytecode compiler) is missing, but it's not particularly difficult to write.
Yes, I've spent some hours figuring that out. It is sitting in the parser extension, slightly different from the builtin one. For a first implementation, I think to re-code anything necessary quite directly in Python. But then, re-coding again gives us ways to become more flexible concerning the parser, if we don't have the restrictions of C any longer. Well, this is a long way to Tipparary.
Not sure if this helps, but it's all I can contribute right now.
This is very helpful, thanks a lot for supporting this project. I'm happy that you are following it, positively. This is how it is meant to be: We are trying a new approach, and we want to do this as support for Python's evolution, trying to find a way to simplify it and to make its development easier and faster. If we succeed, this may become a new way for Python. If not, then we are probably closing a dead end. Both possible results are able to help. All the best -- chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Hello Christian, On Mon, Jan 13, 2003 at 04:00:37AM +0100, Christian Tismer wrote:
This also touches one of my weak points: Is it good to replace C code by something which is depending on another special module like sre? This makes sre into something crucial to this re-implementation. But I'm not sure if this is a good way to go.
As far as performance is concerned, it could be at some stage re-implemented in pure Python. Character handling may be slow but it is a good target for Psyco optimization.
I'm also not sure if it is good to re-implement certain modules using the common Python tricks and optimizations. To some extent, I have the impression that doing it the simple way, basically as done in C, would fit the optimizations of Psyco better. But that is an open question until we get some feedback from Armin Rigo.
That's right; the "simple way" is what Psyco can best work with. It does not prevent the definition of abstract classes that encapsulate the common data structures; on the other hand, these classes (with suitable information like types and mutability) are quite useful for Psyco, as you already mentionned. Armin
Bengt Richter wrote:
Hi, this sounds interesting ;-)
I was wondering how you were planning on identifying concrete tasks and assigning and tracking them, and keeping status visible to support cooperation.
Good question.
Do you already have CVS and bug/issue tracking methodology settled? I.e., such stuff is nuisance overhead or sanity preservation depending on the task, and the development phase, and scale/distribution of effort.
Nothing yet but this mailing list. The whole "project" is about one week old, and it consists of a lot of emails with random ideas, and the insight that there is something in a couple of brains that really wants to materialize and creep out.
Is there one single home page that leads to everything relevant, with some indication of historical vs current vs fire alarm links?
That's the next thing to do, I guess. I'm a bit reluctant to use SourceForge. If I can manage, I'd liek to use my own new server for the CVS, install a Wiki and so on. This will hopefully be figured out during this week, as time permits.
Here are some things that occur to me to think about in making a roadmap for your project. If you would make something like it to reflect your _actual_ project plans, it would help me (and maybe some other lurkers ;-) get an idea of what you are really up to ;-)
[huuge list of heading lines to be filled] Oh well, this is really an awful lot of work to do, and I'm so happy that we are a group of people and can spread some of the work.
Well, that's all the thoughts I have for now ;-)
Once we have some basic pages, your thoughts can be found there, again. all the best -- chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
participants (4)
-
Bengt Richter -
Christian Tismer -
Guido van Rossum -
RIGO Armin