Mailman 3 December 2005 - Python-Dev

Re: [Python-Dev] Documentation about Python's GC, python-dev list messages referenced in Modules/gcmodule.c not reachable anymore
by Weber, Gregoire 07 Dec '05

07 Dec '05

Hi Neil, Hi Tim, Hi Simon, your responsive and valuable answers cleared up most of the open questions and gave a very positive impression to my project leader about the python community and python itself. Thank you very much! I'm trying to get more into the GC implementation before asking more about some still open points. Gregoire

1 0

ast-objects branch created
by "Martin v. Löwis" 06 Dec '05

06 Dec '05

I created http://svn.python.org/projects/python/branches/ast-objects/ You can convert your repository to that branch with svn switch svn+ssh://pythondev@svn.python.org/python/branches/ast-objects in the toplevel directory. In particular, this features http://svn.python.org/projects/python/branches/ast-objects/Parser/asdl_c.py http://svn.python.org/projects/python/branches/ast-objects/Include/Python-a… http://svn.python.org/projects/python/branches/ast-objects/Python/Python-as… The status is currently this: - asdl_c generates a type hierarchy: "Sum" productions have one type per constructor, inheriting from a type for the sum; plain products only have a type for the product. - attributes are in the base type, accessible through o->_base.attr; projections of the product types are accessible directly through member names. - all projections must be non-NULL. Sequences are represented through potentially empty lists; optional types are potentially represented through Py_None. bool is gone; use Py_True/Py_False. The only primitive type remaining is int (which only occurs in lineno) - the types currently have only a constructor, a dealloc function, and an _Check macro. - Naming is this: for cross-object-file visible symbols (functions and global variables), a Py_ prefix is used. Otherwise, I use the type name or constructor name directly. There is a #define for the Py_<type>_New function, so you can also write <type>(params). Parameter order for the types is: projections first, then attributes. - For compatibility with the current code, the Sum base types also have the _kind enumeration (although that appears not to get initialized right now). For asdl_c, I see the following things as TODOs: - add support for traversing the types from C, through tp_members (read-only). Optionally add support for pickling. - add support for garbage collection. I don't expect this to be necessary right now, but will be if the API is exposed, and it doesn't cost much. The bigger chunk of necessary changes is in using these, starting with ast.c. Feel free to commit any changes to that branch that you consider helpful. To avoid duplicated work, posting a note here might also help. Regards, Martin

6 14

c.l.p post on docs
by A.M. Kuchling 06 Dec '05

06 Dec '05

I just posted a lengthy message to comp.lang.python/python-list about Python's docs; the title is "Documentation suggestions". A short summary of the post is: "The RefGuide is hard to read and hard to keep updated. Do we need a friendly language description? If we do that, should the existing RefGuide be abandoned or maintained?" See the post for the full argument. (Hey, the post just showed up in Google Groups: <http://groups.google.com/group/comp.lang.python/browse_frm/thread/767470cb3…>) I suggest further discussion on this issue take place in c.l.py. --amk

1 0

Re: [Python-Dev] [Python-checkins] commit of r41586 - in python/trunk: Lib/SimpleXMLRPCServer.py Misc/NEWS
by Guido van Rossum 06 Dec '05

06 Dec '05

Happened to see this commit. What's the magic about 10MB? Is there any understanding of what causes it to fail? What is the failure mode? Could it just be fragmentation causing the malloc or realloc to fail? Should we perhaps use a more conservative buffer size, e.g. 1MB or even 8K (the international standard for I/O buffering :-)? --Guido On 12/4/05, andrew.kuchling <python-checkins(a)python.org> wrote: > Author: andrew.kuchling > Date: Sun Dec 4 16:36:57 2005 > New Revision: 41586 > > Modified: > python/trunk/Lib/SimpleXMLRPCServer.py > python/trunk/Misc/NEWS > Log: > [Bug #792570] Under Windows, socket.read() seems to run into trouble when > asked to read tens of megabytes of data. On my Mac, it hits MemoryErrors > when reading around 15Mb in one chunk. The fix is to read the body in several > parts, not as one big piece. > > It would be nice to fix the underlying socket.read() problem, too. > > 2.4 bugfix candidate. > > > Modified: python/trunk/Lib/SimpleXMLRPCServer.py > ============================================================================== > --- python/trunk/Lib/SimpleXMLRPCServer.py (original) > +++ python/trunk/Lib/SimpleXMLRPCServer.py Sun Dec 4 16:36:57 2005 > @@ -422,8 +422,19 @@ > """ > > try: > - # get arguments > - data = self.rfile.read(int(self.headers["content-length"])) > + # Get arguments by reading body of request. > + # We read this in chunks to avoid straining > + # socket.read(); around the 10 or 15Mb mark, some platforms > + # begin to have problems (bug #792570). > + max_chunk_size = 10*1024*1024 > + size_remaining = int(self.headers["content-length"]) > + L = [] > + while size_remaining: > + chunk_size = min(size_remaining, max_chunk_size) > + L.append(self.rfile.read(chunk_size)) > + size_remaining -= len(L[-1]) > + data = ''.join(L) > + > # In previous versions of SimpleXMLRPCServer, _dispatch > # could be overridden in this class, instead of in > # SimpleXMLRPCDispatcher. To maintain backwards compatibility, > > Modified: python/trunk/Misc/NEWS > ============================================================================== > --- python/trunk/Misc/NEWS (original) > +++ python/trunk/Misc/NEWS Sun Dec 4 16:36:57 2005 > @@ -451,6 +451,9 @@ > - Bug #1222790: in SimpleXMLRPCServer, set the reuse-address and close-on-exec > flags on the HTTP listening socket. > > +- Bug #792570: SimpleXMLRPCServer had problems if the request grew too large. > + Fixed by reading the HTTP body in chunks instead of one big socket.read(). > + > - Bug #1110478: Revert os.environ.update to do putenv again. > > - Bug #1103844: fix distutils.install.dump_dirs() with negated options. > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > -- --Guido van Rossum (home page: http://www.python.org/~guido/)

3 2

Dynamic Link Library
by musicdev 06 Dec '05

06 Dec '05

Hi everyone, Apologies if my question is a bit novice-ish. I was wondering if there was a way of creating a Dynamic Link Library with Python. Please provide any answers you may have. Warmest Regards, musicdev

2 1

svn problem - can't get log info for a specific revision
by skip＠pobox.com 05 Dec '05

05 Dec '05

I'm investigating some problems we're having at work with SIGUSR1 handling. Our apps tend to see KeyboardInterrupt when sent SIGUSR1 signals. We're still running 2.3.4 but are in the process of installing/testing 2.4.2. A couple "find ... | xargs egrep -i usr1" commands in the 2.3 and 2.4 trees showed me that between the two a test was added to test_threadsignals.py, apparently in svn revision 36760: % svn blame Lib/test/test_threadsignals.py ... 36760 mwh 36760 mwh def registerSignals((for_usr1, for_usr2, for_alrm)): 36760 mwh usr1 = signal.signal(signal.SIGUSR1, for_usr1) 36760 mwh usr2 = signal.signal(signal.SIGUSR2, for_usr2) 36760 mwh alrm = signal.signal(signal.SIGALRM, for_alrm) 36760 mwh return usr1, usr2, alrm 36760 mwh 36760 mwh 41566 fred.drake # The signal handler. Just note that the signal occurred and 36760 mwh # from who. 36760 mwh def handle_signals(sig,frame): 36792 tim_one signal_blackboard[sig]['tripped'] += 1 36760 mwh signal_blackboard[sig]['tripped_by'] = thread.get_ident() 36760 mwh 36760 mwh # a function that will be spawned as a separate thread. 36760 mwh def send_signals(): 36760 mwh os.kill(process_pid, signal.SIGUSR1) 36760 mwh os.kill(process_pid, signal.SIGUSR2) 36760 mwh signalled_all.release() 36760 mwh ... Wanting to investigate that further, I checked the developer's FAQ and found this command to view the checkin history for that rev: svn log --verbose -r 36760 When I run it I get this error: svn: REPORT request failed on '/projects/!svn/bc/36760/python/branches/release24-maint' svn: '/projects/!svn/bc/36760/python/branches/release24-maint' path not found What is that error telling me? I ran the "svn log" command in the same directory as "svn blame" and "svn up", so it's not a basic problem connecting to the server. It appears to be something specific to this command or revision. Thx, Skip

3 4

Proposed additional keyword argument in logging calls
by Vinay Sajip 04 Dec '05

04 Dec '05

On numerous occasions, requests have been made for the ability to easily add user-defined data to logging events. For example, a multi-threaded server application may want to output specific information to a particular server thread (e.g. the identity of the client, specific protocol options for the client connection, etc.) This is currently possible, but you have to subclass the Logger class and override its makeRecord method to put custom attributes in the LogRecord. These can then be output using a customised format string containing e.g. "%(foo)s %(bar)d". The approach is usable but requires more work than necessary. I'd like to propose a simpler way of achieving the same result, which requires use of an additional optional keyword argument in logging calls. The signature of the (internal) Logger._log method would change from def _log(self, level, msg, args, exc_info=None) to def _log(self, level, msg, args, exc_info=None, extra_info=None) The extra_info argument will be passed to Logger.makeRecord, whose signature will change from def makeRecord(self, name, level, fn, lno, msg, args, exc_info): to def makeRecord(self, name, level, fn, lno, msg, args, exc_info, extra_info) makeRecord will, after doing what it does now, use the extra_info argument as follows: If type(extra_info) != types.DictType, it will be ignored. Otherwise, any entries in extra_info whose keys are not already in the LogRecord's __dict__ will be added to the LogRecord's __dict__. Can anyone see any problems with this approach? If not, I propose to post the approach on python-list and then if there are no strong objections, check it in to the trunk. (Since it could break existing code, I'm assuming (please correct me if I'm wrong) that it shouldn't go into the release24-maint branch.) Of course, if anyone can suggest a better way of doing it, I'm all ears :-) Regards, Vinay Sajip

9 13

SVN backup?
by Aahz 04 Dec '05

04 Dec '05

While cleaning up some old CDs, I discovered that I had received some backups of the CVS repository. Should we repeat the exercise for SVN? -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ "Don't listen to schmucks on USENET when making legal decisions. Hire yourself a competent schmuck." --USENET schmuck (aka Robert Kern)

2 1

Proposed additional keyword argument in logging calls
by Jim Jewett 02 Dec '05

02 Dec '05

> I couldn't think of a good reason why it should be possible to overwrite these > values with values from a user-supplied dictionary, other than to spoof log > entries in some way. The intention is to stop a user accidentally overwriting > one of the above attributes. This makes sense, but is it worth the time to check on each logging call? -jJ

2 1

Subject lines of commit email
by "Martin v. Löwis" 02 Dec '05

02 Dec '05

Due to popular demand, we have changed the format of the subject lines that mailer.py will send out on commit, to include the file names of the modified files (not just the directory). As the original mailer.py couldn't do that, David Goodger contributed the necessary changes. If people feel strongly that this should be reverted, we will need to take a poll, as people also feel strongly that file names in the subject line are essential. Regards, Martin

1 0