时间戳

首页 > 解决方案 > 如何使用nodejs删除重复的csv文件行

node.js - 如何使用nodejs删除重复的csv文件行

问题描述

我正在尝试寻找提供此功能的任何简单方法或任何节点包。

我的 csv 数据如下：

"date","time1","height1","time2","height2","time3","height3","time4","height4"
"30 Ogos 2021","05:08","175","11:30","93","17:00","148","23:04","88"
"31 Ogos 2021","06:00","160","12:39","104","18:40","132",,
"30 Ogos 2021","05:08","175","11:30","93","17:00","148","23:04","88"
"31 Ogos 2021","06:00","160","12:39","104","18:40","132",,

不同的行将具有动态的列数。

标签： node.jscsv

解决方案

这个解决方案效率低下，我相信它是 O(n^2)，但如果我们假设您的 CSV 在 javascript 中表示为字符串的二维数组（即string[][]以打字稿表示法），它会起作用。

const table = loadCsv(...); // this line is pseudocode

/**
 * Check if two arrays are the same
 */
function rowCompare(rowA, rowB) {
  if (rowA.length !== rowB.length) return false;
  for (let i = 0; i < rowA.length; i++) {
    if (rowA[i] !== rowB[i]) return false;
  }
  return true;
}

const dedupedTable = table.filter((row, ind, arr) => {
  return ind === arr.findIndex(r => rowCompare(row, r))
});

该dedupedTable变量现在应该包含您的表，并删除了所有重复的行。

推荐阅读