[Tutor] question about operator overloading

[Joel Goldstick]

On Tue, Mar 6, 2012 at 7:05 AM, Albert-Jan Roskam <fomcl at yahoo.com> wrote:
> From: Steven D'Aprano <steve at pearwood.info>
> To: tutor at python.org
> Sent: Tuesday, March 6, 2012 1:58 AM
>
> Subject: Re: [Tutor] question about operator overloading
>
> Alan Gauld wrote:
>
>> On 05/03/12 21:25, Dave Angel wrote:
>>
>>> It's not clear what __add__() should mean for physical files.
>>
>> My guess would be similar to the cat operator in Unix:
>>
>> $ cat file1, file2 > file3
>>
>> is equivalent to
>>
>> file3 = file1 + file2
>>
>> But of course, thats just my interpretation of file addition...
>
> I think that's what Albert-Jan is probably thinking, but the two models are
> not quite the same. I think that what he wants is probably closer to
> something like the fileinput module. I think what he wants is to avoid this:
> -----> First off, thank you all for your replies, including the replies
> after this mail. And sorry for top-posting in an earlier mail
> -----> And yes indeed Steven and Alan, this is what I had in mind.
> for f in (file1, file2, file3, file4):
>     for record in f:
>         process(record)
>
> in favour of this:
>
> all_the_files = file1 + file2 + file3 + file4  # merge file contents
> for record in all_the_files:
>     process(record)
>
> Albert-Jan, am I close? If not, please explain what you are trying to
> accomplish.
> ----> What I had in mind was something like Peter Otten suggested:
> merged = file1 + file2
> merged.save_as(filename)
> Your solution looks intuitive, but won't "all_the_files" become very large
> if file1 through file4 contain, say, 100 billion values each?
>
>
> If the files are small, the easy way is to just read their contents, add
> them together as strings or lists, and then process the lot. But if the
> files are big, or you want to process them on-demand instead of up-front,
> you need an approach similar to fileinput.
> ----> see above. Btw, contrary to what somebody in this thread said, Spss
> files are binary files, not text files.

I chimed in with an assumption that these are were text files.  Since
i wasn't right assuming that, and if I read the discussion correctly
there are two methods being contemplated.  One way is to read a file,
process it, write the result to an output file, read the next file,
process it and append to the output file.  The second method is to
concatinate all of the input files, then open it up and process it.

But, if the spss files aren't text, then I assume they have some
structure that might not be concatinatable.  Not sure if that is a
word.
>
>
> Personally, all these Reader and Append objects make my brain hurt, and I
> hardly ever use operator overloading, except perhaps for numeric types.
> Reader objects, I can just get. But "Append" objects?
>
> This may be useful:
>
> http://steve-yegge.blogspot.com.au/2006/03/execution-in-kingdom-of-nouns.html
> ----> Nice one ;-))
>
> and also itertools:
>
>
> from itertools import chain
> merged = chain(file1, file2, file3, file4)
> for record in merged:
>     process(record)
> ----> Very, *very* useful function, thank you!
> ----> this is (incomplete) code that I created without bothering about
> __add__:
> with SavWriter(mergedSavFileName, varNames, varTypes) as sav_merged:
>   for savFileName in glob.glob("d:/temp/*.sav"):
>     with SavReader(savFileName) as sav_r:
>       header = sav_r.next()
>       for row in sav_r:
>         sav_merged.writerow(row)
> http://code.activestate.com/recipes/577811-python-reader-writer-for-spss-sav-files-linux-mac-/
> ---> Maybe I am making my life too difficult by trying to use __add__?
>
> -- Steven
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>



-- 
Joel Goldstick