nntplib, huge xover object

[Jp Calderone]

On Tue, Apr 01, 2003 at 11:52:52PM -0500, David Sfiligoi wrote:
> Hi,
> 
> I built a small script that use the xover function in the nntplib module.   
> The problem that I came across is that what xover can return a huge tuple 
> when there is 1000s of article in a newsgroup(which is frequent)

  There are a couple options.

  One is to use a library that doesn't store the entire result of the
protocol action in memory, or at least gives you the option not to.  This is
a good solution, except it probably requires that you not use the standard
nntplib, so it is a bit of extra work.

  Another is to limit the number of articles returned by xover.  This is
done very easily, by simply specifying sane values for start and end, and
making repeated calls until the entire dataset has been retrieved,
processing or dumping to secondary storage whatever intermediate results you
receive.  This is all accomplished simply by selecting appropriate values
for "start" and "end" in your code example.

> 
> testxover_resp,testxover_subs = s.xover(start,end)
> 

  Jp

-- 
"The problem is, of course, that not only is economics bankrupt but it has
 always been nothing more than politics in disguise ... economics is a form
 of brain damage."  -- Hazel Henderson
-- 
 up 13 days, 2:00, 3 users, load average: 1.00, 1.00, 1.03