php - 使用 AWS S3 createMultipartUpload (AWS PHP SDK v3) 上传进度统计信息

Question

问题

目前我正在使用 AWS S3 createMultipartUpload 函数将大文件（视频）上传到 AWS S3 存储。这就像一个魅力。

但是，由于将视频上传到 AWS S3 存储需要大量的上传时间，我正在尝试找到一种解决方案来检索上传进度统计信息，例如，uploadBytes 和 totalBytesToUpload 来计算视频上传完成之前的剩余时间完全的。

我在互联网上搜索了数十个 StackOverflow 问题和博客，了解如何实时检索上传文件的这些统计信息，但还没有找到可行的解决方案。

我尝试了什么？

原始代码

$result = $this->s3Client->createMultipartUpload([
            'Bucket'       => $this->bucket,
            'Key'          => $path,
            'StorageClass' => 'STANDARD',
            'ACL'          => 'public-read',
            'Metadata'     => $metaDataValues
        ]);

使用 @http 进度 (GuzzleHttp) 的代码（在https://stackoverflow.com/a/43700547/4686900上找到）

$result = $this->s3Client->createMultipartUpload([
            'Bucket'       => $this->bucket,
            'Key'          => $path,
            'StorageClass' => 'STANDARD',
            'ACL'          => 'public-read',
            'Metadata'     => $metaDataValues,
            '@http' => [
                'progress' => function ($downloadTotalSize, $downloadSizeSoFar, $uploadTotalSize, $uploadSizeSoFar) {
                    $this->loggerService->logError("down [" . $downloadSizeSoFar ."]/[".$downloadTotalSize."] up [".$uploadSizeSoFar."]/[".$uploadTotalSize."]");
                }
            ]
        ]);

此代码在将视频上传到 AWS S3 存储后记录以下输出

[2020-09-29 16:29:45] customLog.ERROR: down [0]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [0]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [0]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [0]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [0]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [0]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [0]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [0]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [0]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [435]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [435]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [435]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [435]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [435]/[0] up [0]/[0] [] []
[2020-09-29 16:29:45] customLog.ERROR: down [435]/[0] up [0]/[0] [] []

在日志中，您会看到统计信息 [0]/[0]，因此没有任何变化。

我试图考虑其他解决方案，例如使用 JavaScript AJAX XHR 函数计算上传字节数，请参阅代码：

$.ajax(
{
//...
xhr: function() {
      let xhr = new window.XMLHttpRequest();
      xhr.upload.addEventListener("progress", function(evt) {
        if (evt.lengthComputable) {
           let percentComplete = (evt.loaded / evt.total) * 100;
           console.log(percentComplete)
        }
      }, false);
  return xhr;
}
// ...
});

但是，此代码仅捕获我在本地和我们自己的服务器之间传输的字节数，而不是在我的本地环境和 AWS S3 存储服务器之间传输的字节数。

完整代码

public function uploadMultipart($file, $path, $metaDataValues = [])
{
        ini_set('max_execution_time', 0);

        $parts = array();
        $result = $this->s3Client->createMultipartUpload([
            'Bucket'       => $this->bucket,
            'Key'          => $path,
            'StorageClass' => 'STANDARD',
            'ACL'          => 'public-read',
            'Metadata'     => $metaDataValues,
            '@http' => [
                'progress' => function ($downloadTotalSize, $downloadSizeSoFar, $uploadTotalSize, $uploadSizeSoFar) {
                    $this->loggerService->logError("down [" . $downloadSizeSoFar ."]/[".$downloadTotalSize."] up [".$uploadSizeSoFar."]/[".$uploadTotalSize."]");
                }
            ]
        ]);

        $uploadId = $result['UploadId'];

        // Upload the file in parts.
        try
        {
            $file = fopen($file, 'r');
            $partNumber = 1;

            while (!feof($file))
            {
                $result = $this->s3Client->uploadPart([
                    'Bucket'     => $this->bucket,
                    'Key'        => $path,
                    'UploadId'   => $uploadId,
                    'PartNumber' => $partNumber,
                    'Body'       => fread($file, 500 * 1024 * 1024),
                ]);

                $parts['Parts'][$partNumber] = [
                    'PartNumber' => $partNumber,
                    'ETag' => $result['ETag'],
                ];

                $partNumber++;
            }
            fclose($file);
        }
        catch (S3Exception $e) {
            $this->s3Client->abortMultipartUpload([
                'Bucket'   => $this->bucket,
                'Key'      => $path,
                'UploadId' => $uploadId
            ]);

            $this->logger->error("Error multipart uploading: " . $e->getMessage() . " on " . $e->getLine() . " for " . $e->getFile());
        }

        // Complete the multipart upload.
        $result = $this->s3Client->completeMultipartUpload([
            'Bucket'   => $this->bucket,
            'Key'      => $path,
            'UploadId' => $uploadId,
            'MultipartUpload' => $parts,
        ]);

        return $result['Location'];
}

问题

如何在本地系统和 AWS S3 存储服务器之间使用 S3 createMultipartUpload 函数或替代方式检索上传的字节数和要上传的总字节数。

ps 我正在使用 PHP 7.2.20、Symfony 3.4 和 AWS SDK for PHP v3

标签： phpsymfonyamazon-s3file-upload