What should I do if the migration tool exits abnormally?
The tool supports checkpoint restart during uploads. For large files, if the process is interrupted or there is a service failure, you can rerun the tool to resume uploading the incomplete files.
If the files that have been migrated successfully to COS are deleted through the console or other methods, will the migration tool upload them again?
No. All the migrated files are recorded in db. The migration tool scans db directory before each migration and the files recorded in db will not be uploaded again. For more information, see Migration Mechanism and Process.
What should I do if the migration fails with the message "403 Access Deny" displayed in the log?
Please ensure that the key information, bucket information, and region information are correct and that you have the necessary permissions. If you are using a sub-account, the parent account should grant the appropriate permissions. For local migration and migration from other cloud storage, you need to have read and write permissions for the bucket. For bucket copy, you also need read permissions for the source bucket. Additionally, check the client's system time. If it deviates significantly from the actual time, the tool-generated signature time may differ too much from the server's system time, causing the server to consider the signature expired and return a failure.
What should I do if the migration to COS from another cloud storage fails with an error message "Read timed out"?
This error occurs when the data download from other cloud storage times out due to insufficient bandwidth. For example, when you migrate overseas data from AWS to COS, "read time out" may occur due to network latency caused by insufficient bandwidth. To solve this problem, you can increase the network bandwidth and test download speed with
wget before migration.What should I do if the migration fails with a message "503 Slow Down" shown in the log?
This error occurs when frequency control is triggered. A limit of 30,000 QPS is imposed on an account in COS. We recommend you decrease the concurrency for small files in configuration. Then run the tool again to resume the migration.
What should I do if the migration fails with the message "404 NoSuchBucket" shown in the log?
Check whether your key, bucket, and region information is correct.
What should I do if an exception occurs with the following message?
How to resolve the issue of not finding the rocksdb JNI library in a Windows environment?
In a Windows environment, the tool needs to be compiled in Microsoft Visual Studio 2015. In case of the above error message, you need to install Visual C++ Redistributable for Visual Studio 2015.
How do I modify the log level?
Modify the file
src/main/resources/log4j.properties by replacing the value of log4j.rootLogger with the log level, such as DEBUG, INFO, and ERROR.What do I do if the /tmp/librocksdbjnixxx.so: ELF file OS ABI invalid error is reported in the Linux environment?
IFUNC needs to be supported on Linux and the binutils version in the running environment should be later than 2.20.
What should I do if a task fails to be fully executed and "java.nio.file.FileSystemLoopException" is reported in error.log?
The exception information in
error.log is similar to:2022-XX-XX XX:XX:XX [ERROR] [main:xxx] [com.qcloud.cos_migrate_tool.task.MigrateLocalTaskExecutor:] [MigrateLocalTaskExecutor.java:183]walk file tree errorjava.nio.file.FileSystemLoopException: /dataseal/xx1/file1at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:294)at java.nio.file.FileTreeWalker.next(FileTreeWalker.java:372)at java.nio.file.Files.walkFileTree(Files.java:2706)at com.qcloud.cos_migrate_tool.task.MigrateLocalTaskExecutor.buildTask(MigrateLocalTaskExecutor.java:176)at com.qcloud.cos_migrate_tool.task.TaskExecutor.run(TaskExecutor.java:244)at com.qcloud.cos_migrate_tool.app.App.main(App.java:135)
The reason is that the file "/dataseal/xx1/file1" to be migrated may be a soft link pointing to a resource in its parent directory. You can check with the following command:
[root@TENCENT64 /dataseal/cos_migrate_tool_v5-master/log]# ll /dataseal/xx1/file1lrwxrwxrwx 1 xx xx xx xx x xxxx /dataseal/xx1/file1 -> ../xx1/
As shown above, the soft link file "/dataseal/xx1/file1" points to its parent directory "/dataseal/xx1/", which will cause an infinite loop during traversal, and the migration task will be automatically terminated. It is recommended to delete such files in advance (Note: Excluding these files in the "excludes" configuration option is ineffective).
If other problems occur, try to run the migration tool again. If the problem persists, please compress the configuration (with the key hidden) as well as the log directory and contact us.