从Azure blob存储中下载所有文件，将其压缩并以JAVA格式上载zip文件
从Azure blob存储中下载所有文件，将其压缩并以JAVA格式上载zip文件
提问于 2021-07-21 19:03:57
我想从Azure blob存储中下载所有文件，从这些文件中创建一个zip文件，然后将该zip文件上传回blob存储。由于文件大小可能非常大，所以我不想最大限度地耗尽内存。此外，此操作需要非常快。
JAVA SDK - azure-storage-blob 12.8.0
编辑:到目前为止编写的代码。不确定如何继续并行上传pipedinputstream数据。
String zipFileName = formFileName(exportRequest, requestId);
final PipedOutputStream pipedOutputStream = new PipedOutputStream();
final PipedInputStream pipedInputStream = new PipedInputStream(pipedOutputStream);
AzureObjectStoreService objectStoreService =managedObjectStoreUtils.getObjectStoreService();
if (filesToZip.size() > 0) {
System.out.println("Files to zip "+ filesToZip.size());
CompletableFuture<Boolean> zipCreationFuture = CompletableFuture.runAsync(() -> {
LoggerHelper.logInfo(logger, "Inside createZIP file async function");
ZipOutputStream zipOutputStream = new ZipOutputStream(pipedOutputStream);
try {
for (String fileName : filesToZip) {
try {
BlobClient blobClient = objectStoreService.getBlobContainerClient().getBlobClient(fileName);
LoggerHelper.logInfo(logger, "Adding zipEntry for file : " + fileName);
final ZipEntry zipEntry = new ZipEntry(fileName);
zipOutputStream.putNextEntry(zipEntry);
byte[] buffer;
ByteArrayOutputStream output = new ByteArrayOutputStream();
buffer= output.toByteArray();
blobClient.getBlockBlobClient().download(output);
int len;
while ((len = buffer.length) > 0) {
zipOutputStream.write(buffer, 0, len);
}
zipOutputStream.closeEntry();
} catch (SdkClientException e) {
LoggerHelper.logExceptionWithMessage(logger, this.getClass().getName(), (Exception) e);
LoggerHelper.logError(logger, "Failed while getting s3 object");
}
}
zipOutputStream.finish();
} catch (IOException ex) {
LoggerHelper.logExceptionWithMessage(logger, this.getClass().getName(), (Exception) ex);
LoggerHelper.logError(logger, "Creating zip file failed");
} finally {
try {
zipOutputStream.close();
} catch (IOException e) {
LoggerHelper.logExceptionWithMessage(logger, this.getClass().getName(), (Exception) e);
LoggerHelper.logError(logger, "Failed to close the zip output stream");
}
}
LoggerHelper.logInfo(logger, "Completed createZIP file async function");
// return true;
}).handle((o, exception) -> {
LoggerHelper.logExceptionWithMessage(logger, this.getClass().getName(), (Exception) exception);
LoggerHelper.logError(logger, "Creating zip file failed");
return null;
});回答 1
Stack Overflow用户
发布于 2021-07-21 22:58:50
能够做到这一点。如果有人有更好的方法，请告诉我。
CompletableFuture.runAsync(() -> {
BlobClient blobClient = objectStoreService.getBlobContainerClient().getBlobClient(zipFileName);
BlobOutputStream blobOutputStream = blobClient.getBlockBlobClient().getBlobOutputStream();
try {
int nextData= pipedInputStream.read();
while (nextData!=-1) {
blobOutputStream.write(nextData);
nextData = pipedInputStream.read();
}blobOutputStream.close();
} catch (IOException e) {
e.printStackTrace();
}页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/68468373
复制相关文章
相似问题
腾讯云开发者
