awk - 如何基于多列 awk 比较 2 个 csv 文件？

Question

我有 2 个样本格式如下的 csv 文件，每个文件有 ~ 5000 行：

文件 1：

    EMPLOYEE_NUMBER,LAST_NAME,FIRST_NAME,MIDDLE_NAME,BRANCH,DEPARTMENT,LEVEL,POSITION,EMAIL_ADDRESS
110426,Balbon,Susan,Lagat,"abc Equity Ventures, Inc.",Group Internal Audit,Supervisor,I.S. Audit Supervisor,[e-mail]
30083,Mendezona,Bingen,Roemer,"abc Equity Ventures, Inc.",Risk Management Office,Vice President,VP - AEV Security,[e-mail]
110773,Casas,Joyce Grace,Bea,"abc Equity Ventures, Inc.",Tax Advisory and Compliance,Manager,Tax Counsel,[e-mail]
286,Fernandez,Mark Brian,Tato,abc Foundation Inc.,Computer Services Division,Supervisor,Senior Applications Supervisor,[e-mail]
291,Plando,Marilou,Polleros,"abc Equity Ventures, Inc.",Administration,Assistant Vice President,AVP - Risk Management,[e-mail]
110813,Gemelo-Abarca,Therese Xyza,Dableo,"abc Equity Ventures, Inc.",Governance & Compliance Team,Manager,Associate General Counsel - Corporate Secretarial and Compliance,[e-mail]
30096,Abay,Joanna Marie,Saluria,"abc Equity Ventures, Inc.",Tax Advisory and Compliance,Supervisor,Tax Compliance Officer,[e-mail]
110711,Ostan,Margilyn,Salibio,"abc Equity Ventures, Inc.",Accounting,Staff,Senior Accountant 1,[e-mail]
110732,Fumar-Gonzales,Vanessa Concepcion,Altarejos,"abc Equity Ventures, Inc.",Legal and Corporate Services,Manager,Associate General Counsel - Labor & Litigation,[e-mail]

文件 2：

    EMPLOYEE_NUMBER,LAST_NAME,FIRST_NAME,MIDDLE_NAME,BRANCH,DEPARTMENT,LEVEL,POSITION,EMAIL_ADDRESS
110426,Balbon,Susan,Lagat,"abc Equity Ventures, Inc.",Group Internal Audit,Supervisor,I.S. Audit Supervisor,[e-mail]
30083,Mendezona,Bingen,Roemer,"abc Equity Ventures, Inc.",Security,Vice President,VP - AEV Security,[e-mail]
110773,Casas,Joyce Grace,Bea,"abc Equity Ventures, Inc.",Tax Advisory and Compliance,Supervisor,Tax Counsel,[e-mail]
286,Fernandez,Mark Brian,Tato,abc Foundation Inc.,Computer Services Division,Supervisor,Senior Applications Supervisor,[e-mail]
291,Plando,Marilou,Polleros,"abc Equity Ventures, Inc.",Risk Management Office,Assistant Vice President,AVP - Risk Management,[e-mail]
110866,Dugan,Belinda,Escultura,"abc Equity Ventures, Inc.",Legal Management,Vice President,Vice President for Legal Services Management,[e-mail]
221,Montehermoso,Gladys,Enoy,"abc Equity Ventures, Inc.",Accounting,Staff,Senior Accountant,[e-mail]
30102,Oblianda,Anna Cielo,Salud,"abc Equity Ventures, Inc.",Accounting,Supervisor,Accounting Supervisor,[e-mail]
110499,Bucol,Charmaine Ann,Rebusa,"abc Equity Ventures, Inc.",Group Internal Audit,Staff,Audit Senior,[e-mail]

我想归档所有行在 EMPLOYEE_NUMBER+EMAIL_ADDRESS 列中具有相同的值，但在使用 awk 的其他列中具有不同的值。

我的理想是基于垂直列 EMPLOYEE_NUMBER+EMAIL_ADDRESS 合并 2 个 csv 文件，并使用 awk 删除重复的行。谢谢

输出将是这样的：

EMPLOYEE_NUMBER,LAST_NAME,FIRST_NAME,MIDDLE_NAME,BRANCH,DEPARTMENT,LEVEL,POSITION,EMAIL_ADDRESS
110773,Casas,Joyce Grace,Bea,"Aboitiz Equity Ventures, Inc.",Tax Advisory and Compliance,Manager,Tax Counsel,[e-mail]
110773,Casas,Joyce Grace,Bea,"Aboitiz Equity Ventures, Inc.",Tax Advisory and Compliance,Supervisor,Tax Counsel,[e-mail]

标签： awk

Answer 1

使用简单的 awk_script 就可以实现，

awk_script：

NR==FNR{
  if(FNR==1){print}
  a[$1 $2]=$0
  next
}
a[$1 $2]!=$0 && a[$1 $2]!=""{
  print a[$1 $2],$0
}

要执行的命令，

awk -F',' -v OFS="\n" -f awk_script file1 file2