Merge/append CSV files with different headers
vincent at vincentdavis.net
Mon Mar 24 22:20:34 CET 2014
Thanks for the feedback.
On Mon, Mar 24, 2014 at 1:44 PM, Chris Angelico <rosuav at gmail.com> wrote:
> On Tue, Mar 25, 2014 at 4:50 AM, Vincent Davis <vincent at vincentdavis.net>
> > I have several csv file I need to append (vertically). They have
> > but overlapping headers. For example;
> > file1 headers ['a', 'b', 'c']
> > file2 headers ['d', 'e']
> > file3 headers ['c', 'd']
> > Is there a better way than this
> Summary of your code:
> 1) Build up a set of all headers used, by opening each file and
> reading the headers.
> 2) Go through each file a second time and write them out.
> That seems like the best approach, broadly. You might be able to
> improve it a bit (it might be tidier to open each file once, but since
> you're using two different CSV readers, it'd probably not be), but by
> and large, I'd say you have the right technique. Your processing time
> here is going to be dominated by the actual work of copying.
> The only thing you might want to consider is order. The headers all
> have a set order to them, and it'd make sense to have the output come
> out as ['a', 'b', 'c', 'd', 'e'] - the first three from the first
> file, then adding in everything from subsequent files in the order
> they were found. Could be done easily enough by using 'in' and
> .append() on a list, rather than using a set. But if that doesn't
> matter to you, or if something simple like "sort the headers
> alphabetically" will do, then I think you basically have what you
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-list