时间戳

首页 > 解决方案 > 如何将 Zip 文件从 s3 存储桶下载到 node.js lambda 函数中的本地目录

javascript - 如何将 Zip 文件从 s3 存储桶下载到 node.js lambda 函数中的本地目录

问题描述

我正在尝试从 s3 获取一个 zip 文件并将其存储在本地目录中，然后读取该目录以确保 zip.file 存在。有没有可靠的方法来做到这一点。当我阅读以查看文件是否位于 tmp 目录中时，它不存在。

// MAKE DIRECTORIES FOR ZIPPED AND UNZIPPED FOLDER
const zipDirFolder = await mkdirp(localZippedFolder, { recursive: true });
const unZipDirFolder = await mkdirp(localUnzippedFolder, { recursive: true });

// VERIFY THAT DIRECTORIES ARE CREATED
const zipFolderChecker = fs.statSync(zipDirFolder);
const unZipFolderChecker = fs.statSync(unZipDirFolder);
console.log('SUCCESS: 2 DIRECTORIES CREATED');

// CREATE STREAMS FOR S3 BUCKET OBJECTS
const tempFileName = path.join(zipDirFolder, 'downloaded.zip');
const tempFileDownload = fs.createWriteStream(tempFileName);

// DOWNLOAD ZIP FILE FROM S3 BUCKET TO TEMP FOLDER
const getS3Object = await s3.getObject(params).createReadStream().pipe(tempFileDownload);
console.log('SUCCESS: ZIP FILE DOWNLOADED');

// LIST FILES IN ZIP FOLDER DIRECTORY
await new Promise((resolve, reject) => {
    fs.readdir(zipDirFolder, (error, files) => {
        error ? reject(error) : resolve(files);
        console.log(files)

任何有关如何修复的示例将不胜感激。

标签： javascriptnode.jsasynchronousamazon-s3lambda

解决方案

基于 AWS 文档：https ://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/requests-using-stream-objects.html 。

您的代码几乎是正确的，等待下载过程结束的问题。您能否在代码中添加调试信息：

const s3Stream = s3.getObject(params).createReadStream();
    s3Stream.on('error', function(err) {
        // NoSuchKey: The specified key does not exist
        console.error(err);
    });
    // there is the main debugging point
    s3Stream.pipe(tempFileDownload).on('error', function(err) {
        // capture any errors that occur when writing data to the file
        console.error('File Stream:', err);
    }).on('close', function() {
        console.log('Done.');
    });

此外，您的流不是在await声明之后而是在on('close', () => {})事件之后关闭。您需要将其转换为承诺：

async getObjectToFile(bucket, key):Promise<void> {     
        return new Promise<void>((res, rej) => {
            // *
            // *
           //your code
s3Stream.pipe(tempFileDownload).on('error', function(err) {
            // capture any errors that occur when writing data to the file
            console.error('File Stream:', err);
            rej(err) ;
        }).on('close', function() {
            console.log('Done.');
            res()
        });

        });
    };

推荐阅读