java - 读取的数据长度与预期不同:dataLength=964481363;预期长度=964481376;
问题描述
我正在尝试从 S3 读取加密对象,然后将其上传到 S3 中具有不同加密密钥的新存储桶。当我尝试读取对象时,使用 getObject() 并尝试使用 PutObject 请求放置对象。
当我打印源的内容长度时,是964481376。那么,数据长度是从哪里来的呢?这是 964481363
我的源文件的元数据如下所示:
内容类型
二进制/八位字节流
x-amz-meta-x-amz-未加密内容长度
964481363
x-amz-meta-x-amz-wrap-alg
公里
我无法共享 KMS 密钥。
public boolean copyS3Object(AmazonS3Encryption sourceS3Client, AmazonS3URI sourceS3Uri,
AmazonS3Encryption destS3Client, AmazonS3URI destS3Uri) throws AmazonServiceException, IOException {
//String eTag;
S3Object sourceS3Object = null;
try {
//Get the Source object stream
sourceS3Object = sourceS3Client.getObject(sourceS3Uri.getBucket(), sourceS3Uri.getKey());
ObjectMetadata objectMetadata = sourceS3Object.getObjectMetadata();
System.out.println("content length " + objectMetadata.getContentLength());
PutObjectRequest putRequest = new PutObjectRequest(destS3Uri.getBucket(), destS3Uri.getKey(),
sourceS3Object.getObjectContent(), sourceS3Object.getObjectMetadata());
destS3Client.putObject(putRequest);
//
// if (eTag.isEmpty()) {
// System.out.println("Copy failed, New object in " + destS3Uri.toString() + " is empty");
// return false;
// }
} catch (AmazonServiceException e) {
throw e;
} finally {
if (sourceS3Object != null) {
sourceS3Object.close();
}
}
//System.out.println("Copied successfully to " + destS3Uri.toString() + " Etag:" + eTag);
return true;
}
但我收到以下错误:
{ "errorMessage": "读取的数据长度与预期不同:dataLength=964481363;expectedLength=964481376;includeSkipped=false;in.getClass()=class com.amazonaws.internal.ReleasableInputStream;markedSupported=false;marked=0 ; resetSinceLastMarked=false; markCount=0; resetCount=0", "errorType": "com.amazonaws.SdkClientException", "stackTrace": ["com.amazonaws.util.LengthCheckInputStream.checkLength(LengthCheckInputStream.java:151)", “com.amazonaws.util.LengthCheckInputStream.read(LengthCheckInputStream.java:109)”、“java.io.FilterInputStream.read(FilterInputStream.java:107)”、“com.amazonaws.services.s3.internal.crypto.CipherLiteInputStream .nextChunk(CipherLiteInputStream.java:225)”,“com.amazonaws.services.s3。internal.crypto.CipherLiteInputStream.read(CipherLiteInputStream.java:118)”、“com.amazonaws.services.s3.internal.crypto.RenewableCipherLiteInputStream.read(RenewableCipherLiteInputStream.java:112)”、“com.amazonaws.internal.SdkFilterInputStream。读取(SdkFilterInputStream.java:90)”,“com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:90)”,“com.amazonaws.util.LengthCheckInputStream.read(LengthCheckInputStream.java:107)”,“com .amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:90)”、“com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream.read(MD5DigestCalculatingInputStream.java:128)”、“java.io.BufferedInputStream.fill(BufferedInputStream .java:246)", "java.io.BufferedInputStream.read1(BufferedInputStream.java:286)”、“java.io.BufferedInputStream.read(BufferedInputStream.java:345)”、“com.amazonaws.internal.SdkBufferedInputStream.read(SdkBufferedInputStream.java:76)”、“com.amazonaws .internal.SdkFilterInputStream.read(SdkFilterInputStream.java:90)"、"com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:180)"、"com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:90) )”、“org.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:140)”、“com.amazonaws.http.RepeatableInputStreamRequestEntity.writeTo(RepeatableInputStreamRequestEntity.java:160)”、“org.apache.http. impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)", "org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:162)”、“org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)”、“com.amazonaws.http. protocol.SdkHttpRequestExecutor.doSendRequest(SdkHttpRequestExecutor.java:63)”, “org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)”, “org.apache.http.impl.execchain.MainClientExec.execute( MainClientExec.java:271)”、“org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)”、“org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java: 184)”、“org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)”、“org.apache.http.impl.client.CloseableHttpClient.执行(CloseableHttpClient.java:55)”、“com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)”、“com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient. java:1297)"、"com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)"、"com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)"、"com .amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)"、"com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)"、"com.amazonaws.http.AmazonHttpClient$RequestExecutor .access$500(AmazonHttpClient.java:686)", "com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)"、"com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)"、"com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512) ", "com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4914)", "com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4860)", "com.amazonaws.services .s3.AmazonS3Client.access$300(AmazonS3Client.java:389)”、“com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:5793)”、“com.amazonaws.services.s3.AmazonS3Client. uploadObject(AmazonS3Client.java:1786)", "com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1746)", "com.amazonaws.services.s3.AmazonS3EncryptionClient.access$101(AmazonS3EncryptionClient.java:81)"、"com.amazonaws.services.s3.AmazonS3EncryptionClient$S3DirectImpl.putObject(AmazonS3EncryptionClient.java:690)"、"com.amazonaws.services .s3.internal.crypto.S3CryptoModuleBase.putObjectUsingMetadata(S3CryptoModuleBase.java:175)”,“com.amazonaws.services.s3.internal.crypto.S3CryptoModuleBase.putObjectSecurely(S3CryptoModuleBase.java:161)”,“com.amazonaws.services .s3.internal.crypto.CryptoModuleDispatcher.putObjectSecurely(CryptoModuleDispatcher.java:108)”、“com.amazonaws.services.s3.AmazonS3EncryptionClient.putObject(AmazonS3EncryptionClient.java:570)”、“com.amazon.testCase.lambda.object .UploadObject.coyS3Object(UploadObject.java:93)”,“com.amazon.testCase。lambda.object.UploadObject.handleRequest(UploadObject.java:54)”、“sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)”、“sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)”、“sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)”。 reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)", "java.lang.reflect.Method.invoke(Method.java:498)"] }
解决方案
我解决了这个问题。S3 对象的内容长度是加密后的数据长度。但是当我放置对象时,我试图放置未加密的数据。所以我需要将对象的内容长度显式设置为未加密的长度。下面的代码解决了这个问题。
try {
//Get the Source object stream
sourceS3Object = sourceS3Client.getObject(sourceS3Uri.getBucket(), sourceS3Uri.getKey());
//For a decrypted object, the content length in metadata has the encrypted length
//Set the content length to the unencrypted-data-length
ObjectMetadata objectMetadata = sourceS3Object.getObjectMetadata();
Map<String, String> modelMetaData = objectMetadata.getUserMetadata();
Long unencryptedDataLength = Long.parseLong(modelMetaData.get(
"x-amz-unencrypted-content-length"));
objectMetadata.setContentLength(unencryptedDataLength);
PutObjectRequest putRequest = new PutObjectRequest(destS3Uri.getBucket(), destS3Uri.getKey(),
sourceS3Object.getObjectContent(), objectMetadata);
eTag = destS3Client.putObject(putRequest).getETag();
if (eTag.isEmpty()) {
System.out.println("Copy failed, New object in " + destS3Uri.toString() + " is empty");
return false;
}
推荐阅读
- oracle - ORA-03001: 将 XMLTYPE 从表转换为 NCLOB 时未实现的功能
- android - 如何从 RecyclerView 中自动选择第一项?
- javascript - JQuery:使用 JQuery 在 iframe 下选择 div
- javascript - 检测 iCheck 复选框上的 Shift 键 + 单击事件
- java - ForkJoinPool 任务优先级
- java - Java:使用列表映射 POST 正文
- javascript - 是否可以在不使用文件路径 src 的情况下插入图像?
- c++ - 如何推导出 std::advance 迭代器类型?
- css - Sidenav 中的右对齐开关
- c# - xamarin 表单 - 使用 c# 计算图像源的尺寸