首页 > 解决方案 > CSV 到 JavaScript 对象 node.js 的日期解析问题 - 零被丢弃

问题描述

概述

微服务 A定期从 SFTP 服务器下载 CSV 并将其上传到 S3 存储桶。CSV 文件(服务器和 S3 存储桶中)中的日期字符串有效:

2018-08-20T10:45:00

微服务 B从 S3 存储桶下载 CSV。使用以下代码将数据转换为 JSON:

   import * as parser from 'csv-parse';

   const streamFromS3 = (params: any) => 
      s3.getObject(params).createReadStream();

  const streamToJSON = (stream: NodeJS.ReadableStream): Promise<any[]> => {
    return new Promise((resolve, reject) => {
      const csvParser = parser({
        relax: true,
        relax_column_count: true,
        columns: true,
        skip_lines_with_error: true,
        skip_empty_lines: true,
        skip_lines_with_empty_values: true,
      });

      const jsonArray: any[] = [];

      stream.pipe(csvParser)
        .on('data', (data: any) => {
        jsonArray.push(data);
      })
      .on('error', (error) => {
        reject(error);
      })
      .on('end', () => {
        resolve(jsonArray);
      });
    });
  };

在此之后,数据被批量写入 MongoDB:

  mapLocationsFromJson = (json: any[]) =>
    Promise.all(json.map(j => this.mapLocation(j)))

mapLocation = (json: any) => {
    const {
      CID: companyId,
      LocID: clinicId,
      Location: clinicName,
      Addr1: address,
      Addr2: address2,
      DateCreated: dateCreated,
      City: city,
      ZipCode: zipcode,
      Phone: phone,
      LocationLongName: longName,
      CoName: companyName,
  } = json;
    // tslint:disable-next-line:max-line-length
    const location = new Location({ clinicId, longName, dateCreated companyId, clinicName, location: { address, address2, city, zipcode }, companyName, phone });
    return location.save();
}

我定期收到以下日期验证错误:

message: 'Cast to Date failed for value "2018-08-2T10:45:00" at path "startTime"',
        name: 'CastError',
        stringValue: '"2018-08-2T10:45:00"',
        kind: 'Date',
        value: '2018-08-2T10:45:00',
        path: 'startTime',
        reason: [Object] } },
  _message: 'Availability validation failed',
  name: 'ValidationError' 

因为日期无效 - 日期字符串现在2018-08-2T10:45:00是,并且在 2 之后缺少结尾 0。

编辑:日期在stream.on('data')块中正确格式化:

2018-08-20T10:45:00

编辑::

我试图手动转换.on('data'块中的每个数据:

stream.pipe(csvParser)
      .on('data', (data: any) => {
        if (data.startTime) {
          const copy = data;
          const formattedDate = moment(data.startTime).toISOString();
          copy.startTime = formattedDate;
          jsonArray.push(copy);
        } else {
          jsonArray.push(data);
        }

仍然没有运气。

为什么 0 被丢弃,我该如何解决这个问题?

标签: javascriptnode.jsmongodbtypescriptamazon-s3

解决方案


推荐阅读