merging several large CSV's

McGlinchy, Alistair Alistair.McGlinchy at
Thu Jun 14 18:37:25 BST 2007

Alex wrote:
> I've been hunting through CPAN for a standard perl module 
> which merges several  large CSVs based on a known key. The 
> nearest I can find is Parse::CSV which lets me stream load 
> single CSVs - but I would then have to implement the merging 
> myself. It seems to me that is a common job.

Maybe your defintion of merge is harder than mine. Modulo the trickiness
of escape conditions for quotes and comma's what's wrong with this?

my $file_one="foo.csv";
my $file_two="bar.csv";
my $key_col_one = 0;  # Column 0 contains the primary key field
my $key_col_two = 2;  # Column 2 contains the foreign key field 

open my $ONE , $file_one or die "Cannot open $file_one : $!"; 
my %store= map { chomp; ( (split/,/)[$key_col_one],  $_ ) } <$ONE>;
close $ONE;

open my $TWO , $file_two or die "Cannot open $file_two : $!"; 
while ( <$TWO> ) {
    my $foreign_key = (split/,/)[$key_col_two];
    print  $store{$foreign_key} , "," , $_, "\n";
close $TWO;

Registered Office:
Marks and Spencer plc
Waterside House
35 North Wharf Road
W2 1NW

Registered No. 214436 in England and Wales.

Telephone (020) 7935 4422
Facsimile (020) 7487 2670


Please note that electronic mail may be monitored.

This e-mail is confidential. If you received it by mistake, please let us know and then delete it from your system; you should not copy, disclose, or distribute its contents to anyone nor act in reliance on this e-mail, as this is prohibited and may be unlawful.

More information about the mailing list