Help & Documentation>Key Management Service>FAQs>Implementing Exponential Backoff to Deal with Service Frequency

Implementing Exponential Backoff to Deal with Service Frequency

Last updated: 2023-08-24 11:47:56

On this page:

Suggestions for Dealing with Exceptions

If exceptional errors occur when you call KMS APIs to send requests from your application to the remote KMS server, you can deal with the errors as suggested below:
Cancel: If the returned error indicates a non-temporary failure or retrying does not lead to success, you should terminate/cancel the program call and report the exception.
Retry: If the returned error is uncommon or relatively rare, such as network packets being damaged during transmission but still sent, you can immediately attempt a retry in this case.
Delayed Retry: If the returned error is due to common connection or busy-related issues, the service may need a short recovery time to clear accumulated workloads. In such cases, wait for an appropriate amount of time before retrying.
This article elaborates on the delayed retry strategy. The mentioned waiting time (i.e., delay time) can be implemented by gradually increasing the delay or using a timed strategy (such as exponential backoff). Since the call frequency to KMS API services is limited, you can use the delayed retry method to avoid issues caused by exceeding the rate limit when your call concurrency is too high.

Exponential Backoff

Pseudocode

// Gradually increase re-execution delays
InitDelayValue = 100
For(Retries = 0; Retries < MAX_RETRIES; Retries = Retries+1)
        wait for (2^Retries * InitDelayValue) milliseconds
        Status = KmsApiRequest()
        IF Status == SUCCESS
            BREAK // Succeeded, stop calling the API again.
        ELSE IF Status = THROTTLED || Status == SERVER_NOT_READY
            CONTINUE // Failed due to throttling or server busy, try again.
        ELSE
            BREAK // another error occurs, stop calling the API again.
        END IF

Policy Implementation

Python: implement exponential backoff for frequency errors in KMS API calls to Encrypt
# -- coding: utf-8 --
import base64
import math
import time
import os
from tencentcloud.common import credential
from tencentcloud.common.exception.tencent_cloud_sdk_exception import TencentCloudSDKException
from tencentcloud.common.profile.client_profile import ClientProfile
from tencentcloud.common.profile.http_profile import HttpProfile
from tencentcloud.kms.v20190118 import kms_client, models


def KmsInit(region="ap-guangzhou", secretId="", secretKey=""):
          try:
                credProfile = credential.Credential(secretId, secretKey)
                client = kms_client.KmsClient(credProfile, region)
                return client
          except TencentCloudSDKException as err:
                print(err)
                return None

def BackoffFunction(RetryCount):
            InitDelayValue = 100
            DelayTime = math.pow(2, RetryCount) * InitDelayValue
            return DelayTime

if __name__ == '__main__':
            # User's Custom Parameters
            secretId = os.getenv('SECRET_ID')         # read from environment variable or use whitebox encryption to protect secret ID
            secretKey = os.getenv('SECRET_KEY')   # read from environment variable or use whitebox encryption to protect secret key    region = "ap-guangzhou"
            region = "ap-guangzhou"
            keyId = "replace-with-realkeyid"
            plaintext = "abcdefg123456789abcdefg123456789abcdefg"
            Retries = 0
            MaxRetries = 10
            client = KmsInit(region, secretId, secretKey)
            req = models.EncryptRequest()
            req.KeyId = keyId
            req.Plaintext = base64.b64encode(plaintext)
            while Retries < MaxRetries:
                try:
                    Retries += 1
                    rsp = client.Encrypt(req)  # Invoke the encryption interface
                    print 'plaintext: ',plaintext,'CiphertextBlob: ',rsp.CiphertextBlob
                    break
                except TencentCloudSDKException as err:
                    if err.code == 'InternalError' or err.code == 'RequestLimitExceeded':
                        if Retries == MaxRetries: 
                            break
                        time.sleep(BackoffFunction(Retries + 1))
                        continue
                    else:
                        print(err)
                        break
                except Exception as err:
                    print(err)
                    break
Note
To deal with other specific errors, you can directly modify the content of the statement except.
Based on your code logic, business strategy, and other factors, plan and establish a timed strategy to set the optimal Initial Delay Value (InitDelayValue) and Retry Count (Retries). This helps avoid setting the threshold too low or too high, which may impact the overall operation of your business.
中国站 - English