Downloading Objects

Last updated: 2023-09-20 14:14:51

Feature Overview

This document provides an overview of APIs and SDK code samples for downloading an object.
API
Operation
Description
Downloading object
Downloads an object to the local file system

Advanced APIs (Recommended)

The advanced API allows you to suspend, resume (via checkpoint restart), or cancel download tasks.
Note
The advanced interface downloads the object directly into the specified local file. If a download stream is required, refer to the simple operation on this page.

Creating a TransferManager instance

Before using the advanced API, first create a TransferManager instance.
// Create a TransferManager instance, which is used to call the advanced API later.
TransferManager createTransferManager() {
// Create a COSClient instance, which is the foundation for accessing COS services.
// For detailed code, refer to this page: Simple Operations -> Create COSClient
COSClient cosClient = createCOSClient();

// Customize the thread pool size. It is recommended to set it to 16 or 32 when the client and COS network are sufficient (e.g., using Tencent Cloud's CVM for same-region COS uploads), which allows for more efficient utilization of network resources.
// For situations where public network transmission is used and the network bandwidth quality is not high, it is recommended to reduce this value to avoid request timeouts due to slow network speed.
ExecutorService threadPool = Executors.newFixedThreadPool(32);

// Pass in a thread pool. If no thread pool is provided, TransferManager will create a single-threaded thread pool by default.
TransferManager transferManager = new TransferManager(cosClient, threadPool);

return transferManager;
}

Description

The TransferManagerConfiguration class is used to record the configuration of advanced APIs. Its main members are described below:
Member Name
Setting Method
Description
Local Disk Types
minimumUploadPartSize
set method
Part size of the multipart upload in bytes. Default: 5 MB
long
multipartUploadThreshold
set method
If a file is greater than or equal to this value, it will be uploaded in concurrent parts. Unit: byte; default: 5 MB
long
multipartCopyThreshold
set method
If a file is greater than or equal to this value, it will be replicated in concurrent parts. Unit: byte; default: 5 GB
long
multipartCopyPartSize
set method
Part size in bytes for multipart replication. Default: 100 MB
long

Closing a TransferManager instance

After confirming that the process does not use the TransferManager instance to call the advanced API anymore, be sure to close it to avoid leaking resources.
void shutdownTransferManager(TransferManager transferManager) {
// If the parameter is set to true, the internal COSClient instance within the transferManager will also be closed.
// If the parameter is set to false, the internal COSClient instance within transferManager will not be closed.
transferManager.shutdownNow(true);
}

Downloading object

This API is used to download a COS object to a local file.

Method prototype

public Download download(final GetObjectRequest getObjectRequest, final File file);

Sample Request

// Before using the advanced API, ensure that the process contains a TransferManager instance. If such an instance does not exist, create one.
// For the detailed code, see "Advanced APIs -> Creating a TransferManager instance" on this page.
TransferManager transferManager = createTransferManager();

// Enter the bucket name in the format of BucketName-APPID.
String bucketName = "examplebucket-1250000000";
// The Object Key is the unique identifier for an object within a bucket. For more information, please refer to [Object Key](https://cloud.tencent.com/document/product/436/13324).
String key = "exampleobject";
// Path to the local file
String localFilePath = "/path/to/localFile";
File downloadFile = new File(localFilePath);

GetObjectRequest getObjectRequest = new GetObjectRequest(bucketName, key);
try {
// Returns an asynchronous Download result, which can be synchronously called using waitForCompletion to wait for the download to finish. On success, it returns void; on failure, it throws an exception.
Download download = transferManager.download(getObjectRequest, downloadFile);
download.waitForCompletion();
} catch (CosServiceException e) {
e.printStackTrace();
} catch (CosClientException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}

// After confirming that the transferManager instance is no longer in use by the current process, close it.
// For the detailed code, see "Advanced APIs -> Closing a TransferManager instance" on this page.
shutdownTransferManager(transferManager);

Description

Parameter name
Description
Local Disk Types
Default value
getObjectRequest
Object download request
GetObjectRequest
-
file
Destination file
File
-
Description of the Request member:
Request Member
Setting Method
Description
Local Disk Types
bucketName
Constructor or set method
Bucket name in the format: BucketName-APPID. For details, see Naming Conventions
String
key
Constructor or set method
The object key is the unique identifier of the object in the bucket. For example, in the object's access domain name examplebucket-1250000000.cos.ap-guangzhou.myqcloud.com/do/picture.jpg, the object key is doc/picture.jpg. For details, see ObjectKey
String
range
set method
Download range
Long[]
trafficLimit
set method
Traffic limits (in bit/s) on the downloaded object. There is no limit by default.
int

Returned value

Success: returns Download. You can query whether the download is complete, or wait until the download is finished.
Failure: an error (such as authentication failure) occurs, with a CosClientException or CosServiceException exception. For details, see Troubleshooting.

Downloading an object (checkpoint restart)

The breakpoint download feature utilizes multi-threaded downloading with specified ranges to download an object and performs integrity verification upon completion. If an interruption occurs during the download process, the already downloaded ranges will not be downloaded again when resuming (unless the source file has been modified before restarting, in which case the download will start from the beginning). This method is suitable for downloading large files.

Method prototype

public Download download(final GetObjectRequest getObjectRequest, final File file,
boolean resumableDownload);

public Download download(final GetObjectRequest getObjectRequest, final File file,
boolean resumableDownload, String resumableTaskFile,
int multiThreadThreshold, int partSize);

Sample Request

// Before using the advanced API, ensure that the process contains a TransferManager instance. If such an instance does not exist, create one.
// For the detailed code, see "Advanced APIs -> Creating a TransferManager instance" on this page.
TransferManager transferManager = createTransferManager();

// Enter the bucket name in the format of BucketName-APPID.
String bucketName = "examplebucket-1250000000";
// The Object Key is the unique identifier for an object within a bucket. For more information, please refer to [Object Key](https://cloud.tencent.com/document/product/436/13324).
String key = "exampleobject";
// Path to the local file
String localFilePath = "/path/to/localFile";
File downloadFile = new File(localFilePath);

GetObjectRequest getObjectRequest = new GetObjectRequest(bucketName, key);
try {
// Returns an asynchronous Download result, which can be synchronously called using waitForCompletion to wait for the download to finish. On success, it returns void; on failure, it throws an exception.
Download download = transferManager.download(getObjectRequest, downloadFile, true);
download.waitForCompletion();
} catch (CosServiceException e) {
e.printStackTrace();
} catch (CosClientException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}

// After confirming that the transferManager instance is no longer in use by the current process, close it.
// For the detailed code, see "Advanced APIs -> Closing a TransferManager instance" on this page.
shutdownTransferManager(transferManager);

Description

Parameter name
Description
Local Disk Types
Default value
getObjectRequest
Object download request
GetObjectRequest
-
file
Destination file
File
-
resumableDownload
Whether to enable checkpoint restart for the download
boolean
false
resumableTaskFile
Name of the file that records the checkpoint restart information
String
file.cosresumabletask
multiThreadThreshold
Minimum file size to use multi-thread download with checkpoint restart
int
20 * 1024 * 1024
partSize
Part size for downloading with checkpoint restart
int
8 * 1024 * 1024
Description of the Request member:
Request Member
Setting Method
Description
Local Disk Types
bucketName
Constructor or set method
Bucket name in the format: BucketName-APPID. For details, see Naming Conventions
String
key
Constructor or set method
The object key is the unique identifier of the object in the bucket. For example, in the object's access domain name examplebucket-1250000000.cos.ap-guangzhou.myqcloud.com/do/picture.jpg, the object key is doc/picture.jpg. For details, see ObjectKey
String
range
set method
Download range
Long[]
trafficLimit
set method
Traffic limits (in bit/s) on the downloaded object. There is no limit by default.
int

Returned value

Success: returns Download. You can query whether the download is complete, or wait until the download is finished.
Failure: an error (such as authentication failure) occurs, with a CosClientException or CosServiceException exception. For details, see Troubleshooting.

Displaying the download progress

You need to configure a function to display the download progress. The function is supposed to call the API to get the object size that has been successfully downloaded and then calculate the current download progress.

Method prototype

public Download download(final GetObjectRequest getObjectRequest, final File file);

Sample Request

// You can adjust the following sample code as needed to form your own code.
void showTransferProgress(Transfer transfer) {
System.out.println(transfer.getDescription());

// transfer.isDone() checks if the download is completed
while (transfer.isDone() == false) {
try {
// Retrieve progress every 2 seconds
Thread.sleep(2000);
} catch (InterruptedException e) {
return;
}

TransferProgress progress = transfer.getProgress();
long sofar = progress.getBytesTransferred();
long total = progress.getTotalBytesToTransfer();
double pct = progress.getPercentTransferred();
System.out.printf("upload progress: [%d / %d] = %.02f%%\n", sofar, total, pct);
}

// Completed successfully, or Failed
System.out.println(transfer.getState());
}
The sample code for the file download operation is as follows:
// Before using the advanced API, ensure that the process contains a TransferManager instance. If such an instance does not exist, create one.
// For the detailed code, see "Advanced APIs -> Creating a TransferManager instance" on this page.
TransferManager transferManager = createTransferManager();

// Enter the bucket name in the format of BucketName-APPID.
String bucketName = "examplebucket-1250000000";
// The Object Key is the unique identifier for an object within a bucket. For more information, please refer to [Object Key](https://cloud.tencent.com/document/product/436/13324).
String key = "exampleobject";
// Path to the local file
String localFilePath = "/path/to/localFile";
File downloadFile = new File(localFilePath);

GetObjectRequest getObjectRequest = new GetObjectRequest(bucketName, key);
try {
// Returns an asynchronous Download result, which can be synchronously called using waitForCompletion to wait for the download to finish. On success, it returns void; on failure, it throws an exception.
Download download = transferManager.download(getObjectRequest, downloadFile);
// Print the download progress until the download is complete
showTransferProgress(download);
// Capture any potential exceptions here
download.waitForCompletion();
} catch (CosServiceException e) {
e.printStackTrace();
} catch (CosClientException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}

// After confirming that the transferManager instance is no longer in use by the current process, close it.
// For the detailed code, see "Advanced APIs -> Closing a TransferManager instance" on this page.
shutdownTransferManager(transferManager);

Description of progress obtaining

You can use the getProgress method of the download class to obtain the TransferProgress class, which has the following three methods to obtain the download progress:
Method Name
Description
Local Disk Types
getBytesTransferred
Obtains the number of bytes downloaded
long
getTotalBytesToTransfer
Obtains the total number of bytes of the file
long
getPercentTransferred
Obtains the percentage of the number of bytes downloaded
double

Description

Parameter name
Description
Local Disk Types
Default value
getObjectRequest
Object download request
GetObjectRequest
-
file
Destination file
File
-
Description of the Request member:
Request Member
Setting Method
Description
Local Disk Types
bucketName
Constructor or set method
Bucket name in the format: BucketName-APPID. For details, see Naming Conventions
String
key
Constructor or set method
The object key is the unique identifier of the object in the bucket. For example, in the object's access domain name examplebucket-1250000000.cos.ap-guangzhou.myqcloud.com/do/picture.jpg, the object key is doc/picture.jpg. For details, see ObjectKey
String
range
set method
Download range
Long[]
trafficLimit
set method
Traffic limits (in bit/s) on the downloaded object. There is no limit by default.
int

Returned value

Success: returns Download. You can query whether the download is complete, or wait until the download is finished.
Failure: an error (such as authentication failure) occurs, with a CosClientException or CosServiceException exception. For details, see Troubleshooting.

Suspending, resuming, or cancelling a download

This API can be used to suspend, resume, or cancel a download task.

Method prototype

public Download download(final GetObjectRequest getObjectRequest, final File file);

Sample Request

// Before using the advanced API, ensure that the process contains a TransferManager instance. If such an instance does not exist, create one.
// For the detailed code, see "Advanced APIs -> Sample code: Creating a TransferManager instance" on this page.
TransferManager transferManager = createTransferManager();

// Enter the bucket name in the format of BucketName-APPID.
String bucketName = "examplebucket-1250000000";
// The Object Key is the unique identifier for an object within a bucket. For more information, please refer to [Object Key](https://cloud.tencent.com/document/product/436/13324).
String key = "exampleobject";
// Path to the local file
String localFilePath = "/path/to/localFile";
File downloadFile = new File(localFilePath);
GetObjectRequest getObjectRequest = new GetObjectRequest(bucketName, key);

try {
// Returns an asynchronous result (copy), which can be synchronously called using waitForCompletion to wait for the download to finish. On success, it returns void; on failure, it throws an exception.
Download download = transferManager.download(getObjectRequest, downloadFile);
// Wait for 3 seconds, then download a portion
Thread.sleep(3000L);
// Pause download and obtain a PersistableUpload instance for subsequent resumption
PersistableDownload persistableDownload = download.pause();
// Complex pause and resume usage:
// PersistableDownload instances can be serialized and stored, then deserialized later to resume the download.
// persistableDownload.serialize(out);
// Continue downloading
download = transferManager.resumeDownload(persistableDownload);
// Capture potential exceptions
download.waitForCompletion();

// Or directly cancel this download
// upload.abort();
} catch (CosServiceException e) {
e.printStackTrace();
} catch (CosClientException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}

// After confirming that the transferManager instance is no longer in use by the current process, close it.
// For the detailed code, see "Advanced APIs -> Sample code: Closing a TransferManager instance" on the current page.
shutdownTransferManager(transferManager);

Description

Parameter name
Description
Local Disk Types
Default value
getObjectRequest
Object download request
GetObjectRequest
-
file
Destination file
File
-
Description of the Request member:
Request Member
Setting Method
Description
Local Disk Types
bucketName
Constructor or set method
Bucket name in the format: BucketName-APPID. For details, see Naming Conventions
String
key
Constructor or set method
The object key is the unique identifier of the object in the bucket. For example, in the object's access domain name examplebucket-1250000000.cos.ap-guangzhou.myqcloud.com/do/picture.jpg, the object key is doc/picture.jpg. For details, see ObjectKey
String
range
set method
Download range
Long[]
trafficLimit
set method
Traffic limits (in bit/s) on the downloaded object. There is no limit by default.
int

Returned value

Success: returns Download. You can query whether the download is complete, or wait until the download is finished.
Failure: an error (such as authentication failure) occurs, with a CosClientException or CosServiceException exception. For details, see Troubleshooting.

Downloading a directory

This API is used to download COS objects that have a specified prefix (a virtual directory) to a specified local directory. The downloaded files are in the same directory structure as in COS.

Method prototype

public MultipleFileDownload downloadDirectory(String bucketName, String keyPrefix,
File destinationDirectory) {

Sample Request

// Before using the advanced API, ensure that the process contains a TransferManager instance. If such an instance does not exist, create one.
// For the detailed code, see "Advanced APIs -> Sample code: Creating a TransferManager instance" on this page.
TransferManager transferManager = createTransferManager();
// Enter the bucket name in the format of BucketName-APPID.
String bucketName = "examplebucket-1250000000";

// Set the prefix of objects to download (similar to downloading a directory in COS). If this parameter is set to "", the entire bucket will be downloaded.
String cos_path = "/prefix";
// Absolute path of the destination folder
String dir_path = "/to/mydir";

try {
// Returns an asynchronous result 'download', which can be synchronously called using 'waitForUploadResult' to wait for the download to complete.
MultipleFileDownload download = transferManager.downloadDirectory(bucketName, cos_path, new File(dir_path));

// You can choose to view the download progress
showTransferProgress(download);

// Or wait for completion in a blocking manner
download.waitForCompletion();

System.out.println("download directory done.");
} catch (CosServiceException e) {
e.printStackTrace();
} catch (CosClientException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}

// After confirming that the transferManager instance is no longer in use by the current process, close it.
// For the detailed code, see "Advanced APIs -> Sample code: Closing a TransferManager instance" on the current page.
shutdownTransferManager(transferManager);

Description

Parameter name
Description
Local Disk Types
bucketName
Name of the bucket in COS
GetObjectRequest
keyPrefix
Prefix of the objects in COS
String
destinationDirectory
Absolute path of the destination directory
File

Returned value

Success: returns MultipleFileUpload. You can query whether the download is complete, or wait until the download is finished.
Failure: an error (such as authentication failure) occurs, with a CosClientException or CosServiceException exception. For details, see Troubleshooting.

Simple Operations

Requests for simple operations need to be initiated through COSClient instances. You need to create a COSClient instance before performing simple operations.
COSClient instances are concurrency-safe. You are advised to create only one COSClient instance for a process and then close it when it is no longer used to initiate requests.

Creating a COSClient instance

Before calling the COS API, first create a COSClient instance.
// Create a COSClient instance, which is used to initiate requests later.
COSClient createCOSClient() {
// Set user identity information.
// To view and manage SECRETID and SECRETKEY, please log in to the Access Management Console at https://console.cloud.tencent.com/cam/capi
String secretId = System.getenv("secretId"); // User's SecretId, it is recommended to use a sub-account key, following the principle of least privilege to reduce usage risks. For obtaining a sub-account key, please refer to https://cloud.tencent.com/document/product/598/37140
String secretKey = System.getenv("secretKey"); // User's SecretKey, it is recommended to use a sub-account key, following the principle of least privilege to reduce usage risks. For obtaining a sub-account key, please refer to https://cloud.tencent.com/document/product/598/37140
COSCredentials cred = new BasicCOSCredentials(secretId, secretKey);

// ClientConfig contains the client settings for subsequent requests to COS:
ClientConfig clientConfig = new ClientConfig();

// Set the bucket region
// For COS_REGION, please refer to https://cloud.tencent.com/document/product/436/6224
clientConfig.setRegion(new Region("COS_REGION"));

// Set the request protocol, either HTTP or HTTPS
// For versions 5.6.53 and below, it is recommended to use the HTTPS protocol.
// Version 5.6.54 and higher, HTTPS is used by default.
clientConfig.setHttpProtocol(HttpProtocol.https);

// The following settings are optional:

// Set socket read timeout, default is 30 seconds
clientConfig.setSocketTimeout(30*1000);
// Set connection establishment timeout, default is 30 seconds
clientConfig.setConnectionTimeout(30*1000);

// Set the HTTP proxy, IP, and port if needed.
clientConfig.setHttpProxyIp("httpProxyIp");
clientConfig.setHttpProxyPort(80);

// Create a COS client.
return new COSClient(cred, clientConfig);
}

Creating a COSClient instance with a temporary key

To request COS using temporary keys, create a COSClient with the temporary keys. This SDK cannot generate temporary keys; additional operations are required for generating them. For more information, please refer to Temporary Key Generation.

// Create a COSClient instance, which is used to initiate requests later.
COSClient createCOSClient() {
// Here, the temporary key must have already been obtained.
// For generating temporary keys, refer to https://cloud.tencent.com/document/product/436/14048#cos-sts-sdk
String tmpSecretId = "TMPSECRETID";
String tmpSecretKey = "TMPSECRETKEY";
String sessionToken = "SESSIONTOKEN";

COSCredentials cred = new BasicSessionCredentials(tmpSecretId, tmpSecretKey, sessionToken);

// ClientConfig contains the client settings for subsequent requests to COS:
ClientConfig clientConfig = new ClientConfig();

// Set the bucket region
// For COS_REGION, please refer to https://cloud.tencent.com/document/product/436/6224
clientConfig.setRegion(new Region("COS_REGION"));

// Set the request protocol, either HTTP or HTTPS
// For versions 5.6.53 and below, it is recommended to use the HTTPS protocol.
// Version 5.6.54 and higher, HTTPS is used by default.
clientConfig.setHttpProtocol(HttpProtocol.https);

// The following settings are optional:

// Set socket read timeout, default is 30 seconds
clientConfig.setSocketTimeout(30*1000);
// Set connection establishment timeout, default is 30 seconds
clientConfig.setConnectionTimeout(30*1000);

// Set the HTTP proxy, IP, and port if needed.
clientConfig.setHttpProxyIp("httpProxyIp");
clientConfig.setHttpProxyPort(80);

// Create a COS client.
return new COSClient(cred, clientConfig);
}

Downloading object

This API (GET Object) is used to download an object to the local host.
Note
The following example demonstrates downloading and using a stream. If you need to download to a file, use the advanced interface on this page.

Method prototype

public COSObject getObject(GetObjectRequest getObjectRequest)
throws CosClientException, CosServiceException;

Sample Request

// Before using the COS API, ensure that the process contains a COSClient instance. If such an instance does not exist, create one.
// For the detailed code, see "Simple Operations -> Creating a COSClient instance" on the current page.
COSClient cosClient = createCOSClient();

// Enter the bucket name in the format of BucketName-APPID.
String bucketName = "examplebucket-1250000000";
// The Object Key is the unique identifier for an object within a bucket. For more information, please refer to [Object Key](https://cloud.tencent.com/document/product/436/13324).
String key = "exampleobject";

GetObjectRequest getObjectRequest = new GetObjectRequest(bucketName, key);
COSObjectInputStream cosObjectInput = null;

try {
COSObject cosObject = cosClient.getObject(getObjectRequest);
cosObjectInput = cosObject.getObjectContent();
} catch (CosServiceException e) {
e.printStackTrace();
} catch (CosClientException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}

// Process the stream for the download.
// Here, the stream is read directly. You can process it according to the actual situation.
byte[] bytes = null;
try {
bytes = IOUtils.toByteArray(cosObjectInput);
} catch (IOException e) {
e.printStackTrace();
} finally {
// Be sure to call close() after using the stream
cosObjectInput.close();
}

// Do not close the COSClient instance before the stream processing is completed.
// After confirming that the current process no longer uses the cosClient instance, close it.
cosClient.shutdown();

Description

Parameter name
Description
Local Disk Types
getObjectRequest
File download request
GetObjectRequest
destinationFile
The file saved locally
File
Description of the Request member:
Request Member
Setting Method
Description
Local Disk Types
bucketName
Constructor or set method
Bucket naming format is BucketName-APPID. For details, see Naming Conventions
String
key
Constructor or set method
The object key is the unique identifier of the object in the bucket. For example, in the object's access domain name examplebucket-1250000000.cos.ap-guangzhou.myqcloud.com/do/picture.jpg, the object key is doc/picture.jpg. For details, see ObjectKey
String
range
set method
Download range
Long[]
trafficLimit
set method
Traffic limits (in bit/s) on the downloaded object. There is no limit by default.
Int

Response description

Success: returns the COSObject class, including the input stream and object attributes.
Failure: an error (such as authentication failure) occurs, with a CosClientException or CosServiceException exception. For details, see Troubleshooting.

Response parameter description

The COSObject class is used to return request results, and includes the main members as outlined below:
Member
Description
Local Disk Types
bucketName
Bucket naming format is BucketName-APPID. For details, see Naming Conventions
String
key
The object key is the unique identifier of the object in the bucket. For example, in the object's access domain name examplebucket-1250000000.cos.ap-guangzhou.myqcloud.com/do/picture.jpg, the object key is doc/picture.jpg. For details, see ObjectKey
String
metadata
Metadata of an object
ObjectMetadata
objectContent
Data stream containing COS object content
COSObjectInputStream