guppy

[Benjamin Kaplan]

On Wed, Nov 9, 2011 at 2:08 PM, Juan Declet-Barreto
<Juan.Declet-Barreto at mesaaz.gov> wrote:
>
> I am trying to build guppy on Python 2.5, but am getting an "initializer element is not constant" error from gcc. I have found very little on this issue in the fora when looking for the general cause of the error; there is even less that is specific to a guppy build on Python 2.5.
>
> One recommendation I have seen is recompiling Python, but apparently that brings up the possibility of rendering it incompatible with non-standard packages like arcpy or arcgisscripting, which are critical to my application.
>
> I am using Cygwin.
>
>

Mixing Cygwin libraries and normal Windows libraries usually doesn't
always work too well. Are you using a Cygwin-compiled version of
Python?

>
> -----Original Message-----
> From: Dominic Binks [mailto:dbinks at codeaurora.org]
> Sent: Wednesday, November 09, 2011 9:31 AM
> To: Juan Declet-Barreto
> Subject: Re: memory management
>
> On 11/9/2011 6:53 AM, Juan Declet-Barreto wrote:
> > After some exception catching, I have found that my program is throwing a MemoryError exception numerous times (~7 iterations of the main loop that processes list elements) until python25\python.exe crashes (Windows XP environment).   I implemented Dave Angel's suggestions re: processing each list element (a file) as I get it.
> >
> > So my question would be how to begin troubleshooting the conditions under which the exception is raised. Is there any more information in the exception object about how much memory is being consumed, or any other indicators as to what led to the exception?
> >
> > -juan
> >
> > -----Original Message-----
> > From: Dave Angel [mailto:d at davea.name]
> > Sent: Monday, November 07, 2011 1:50 PM
> > To: Juan Declet-Barreto
> > Cc: python-list at python.org
> > Subject: Re: memory management
> >
> > On 11/07/2011 03:33 PM, Juan Declet-Barreto wrote:
> >> Well, I am using Python 2.5 (and the IDLE shell) in Windows XP, which ships with ESRI's ArcGIS. In addition, I am using some functions in the arcgisscripting Python geoprocessing module for geographic information systems (GIS) applications, which can complicate things. I am currently isolating standard library Python code (e.g., os.walk()) from the arcgisscripting module to evaluate in which module the environment crash is occurring.
> > You top-posted.  In this mailing list, one should type new information after the quoted information, not before.
> >
> > Perhaps a pre-emptive strike is in order.  On the assumption that it may be a memory problem, how about you turn the app inside out.  Instead of walking the entire tree, getting a list with all the paths, and then working on the list, how about doing the work on each file as you get it.  Or even make your own generator from os.walk, so the app can call your logic on each file, and never have all the file (name)s in memory at the same time.
> >
> >
> > Generator:
> >
> > def  filelist(top, criteria):
> >         for  a, b, c in os.walk():
> >                for fiile in files:
> >                      apply some criteria
> >                      yield file
> >
> >
> > Now the main app can iterate through this "list" in the usual way
> >
> > for filename in filelist(top, "*.txt"):
> >          dosomething...
> >
> >
> >
>
> Look for a tool called heapy.  I had a process that was consuming 10+G of RAM to run.  Using heapy I identified the culprit and reduced down to a more manageable 400M.
>
> Dominic
>
> --
> Dominic Binks: dbinks at codeaurora.org
> Employee of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum
> --
> http://mail.python.org/mailman/listinfo/python-list