Help & Documentation>Cloud Object Storage>Tool Guide>FTP Server

FTP Server

Last updated: 2023-09-07 12:27:41

On this page:

Feature Overview

COS FTP Server allows you to directly operate on COS objects and directories over FTP protocol, including uploading/downloading/deleting files and creating folders. This tool is developed with Python, which makes the installation easier.

Note

Upload Mechanism: Stream-based upload without saving to local disk. Simply configure the working directory according to the standard FTP protocol, without occupying actual disk storage space. Download Mechanism: Directly stream data back to the client. Directory Mechanism: The bucket serves as the root directory for the entire FTP Server, and multiple subdirectories can be created under the bucket. Multi-bucket Binding: Supports binding to multiple buckets simultaneously.
Note
Multi-bucket Binding: This is achieved by using different FTP Server working paths (home_dir). Therefore, when specifying different buckets and user information, you must ensure that the home_dir is unique.
Delete Operation Restriction: In the new FTP Server, you can configure the delete_enable option for each FTP user to indicate whether the user is allowed to delete files. Supported FTP Commands: put, mput, get, rename, delete, mkdir, ls, cd, bye, quit, size. Unsupported FTP Commands: append, mget (native mget command is not supported, but batch downloading is still possible in some Windows clients, such as FileZilla client.)
Note
The FTP Server tool currently does not support checkpoint restart functionality.

Getting Started

System environment

OS: Linux. The CVM of Tencent CentOS series is recommended. Windows systems are not supported for now.
psutil-dependent Linux package: python-devel or python-dev, depending on the names of different Linux distributions. It is added using Linux package manager, such as yum install python-devel or aptitude install python-dev.
Python interpreter version: Python 2.7. For more information on the installation and configuration, see Python.
Note
FTP Server tool does not support Python 3.
Dependent packages:
cos-python-sdk-v5 （≥1.6.5）
pyftpdlib （≥1.5.2）
psutil（>=5.6.1）

Usage Limits

Applicable to COS XML version.

Installation and Operation

Download the FTP Server tool at cos-ftp-server. The installation steps are as follows:
1. Enter the FTP Server directory, and run setup.py to install FTP Server and its dependent libraries (network required).
python setup.py install   # Your account should use sudo or have the root permission.
2. Copy the configuration sample file conf/vsftpd.conf.example and name it conf/vsftpd.conf. See Configuration File of this document to correctly configure bucket and user information.
3. Run ftp_server.py to start FTP Server.
python ftp_server.py
FTP Server can also be started in the following two ways:
Execute the nohup command to start it in the backend process:
nohup python ftp_server.py >> /dev/null 2>&1 &
Run the command in the background using the screen command (requires the installation of the screen tool):
screen -dmS ftp
screen -r ftp
python ftp_server.py
#Use the keyboard shortcut Ctrl + A + D to go back to the main screen.

Stop Operation

If you are running the FTP Server directly or using the screen method to run it in the background, you can use the shortcut key Ctrl+C to stop the FTP Server.
If FTP Server is started with the nohup command, you can stop it by the following way:
ps -ef | grep python | grep ftp_server.py | grep -v grep | awk '{print $2}' | xargs -I{} kill {}

Configuration File

The configuration sample file for FTP Server is conf/vsftpd.conf.example. Copy and name it vsftpd.conf, and then configure it as follows:
[COS_ACCOUNT_0]
cos_secretid = COS_SECRETID    # Replaced with your SECRETID
cos_secretkey = COS_SECRETKEY  # Replaced with your SECRETKEY
cos_bucket = examplebucket-1250000000
cos_region = region   # Replaced with your bucket region
cos_protocol = https
#cos_endpoint = region.myqcloud.com
home_dir = /home/user0      # Replaced with the local path you want the FTP to mount to (which should be an actual existing path but not a soft link)
ftp_login_user_name=user0   # Replaced with a custom username
ftp_login_user_password=pass0   # Replaced with a custom password
authority=RW                # The user’s read and write permissions. R: read; W: write; RW: both.
delete_enable=true # true allows the FTP user to delete files by default; false prohibits the user to delete files.

[COS_ACCOUNT_1]
cos_secretid = COS_SECRETID    # Replaced with your SECRETID
cos_secretkey = COS_SECRETKEY  # Replaced with your SECRETKEY
cos_bucket = examplebucket-1250000000
cos_region = region   # Replaced with your bucket region
cos_protocol = https
#cos_endpoint = region.myqcloud.com
home_dir = /home/user1     # Replaced with the local path you want the FTP to mount to (which should be an existing path but not a soft link)
ftp_login_user_name=user1   # Replaced with a custom username
ftp_login_user_password=pass1   #Replace with a user-defined password
authority=RW                # The user’s read and write permissions. R: read; W: write; RW: both.
delete_enable=false # true allows the FTP user to delete files by default; false prohibits the user to delete files.

[NETWORK]
# If the FTP Server is behind a gateway or NAT, you can use this configuration item to specify the IP address or domain name of the gateway for FTP.
masquerade_address = XXX.XXX.XXX.XXX
# The default listening port for the FTP Server is 2121. Ensure that the firewall allows traffic through this port (e.g., if you deploy the FTP Server tool on a Tencent Cloud CVM, you need to open this port in the CVM security group).
listen_port = 2121			
# passive_port allows you to set the port range for passive mode, which defaults to the range [60000, 65535]. Be sure to open these ports in the firewall (e.g., CVM security group).
passive_port = 60000,65535      


[FILE_OPTION]
# The default maximum single file size is up to 200 GB; it is not recommended to set it too large.
single_file_max_size = 21474836480

[OPTIONAL]
# For the following settings, it is recommended to keep the default values unless there is a specific need. If you need to set a value, please enter a reasonable integer.
min_part_size       = default
upload_thread_num   = default
max_connection_num  = 512
max_list_file       = 10000                # The maximum number of files to be listed by ls command. It is not recommended to go beyond this limit. Otherwise, high latency of ls command will occur.
log_level           = INFO                 # Set the log output level.
log_dir             = log                  # Set the directory to store logs. Default: log under the FTP Server directory.
Note
To bind each user to a different bucket, simply add a [COS_ACCOUNT_X] section. For each different COS_ACCOUNT_X section, the following explanations apply:
The username (ftp_login_user_name) and the home directory (home_dir) under each account must be unique, and the home directory must be a directory that exists in the system.
The number of users logging in to each COS FTP Server simultaneously cannot exceed 100.
endpoint and region will not take effect at the same time. To use the public cloud COS service, enter the region field correctly. The endpoint is commonly used in the privatized deployment environment. When both 'region and endpoint are entered, endpoint` will take precedence.
The OPTIONAL settings in the configuration file are provided for advanced users to adjust upload performance. By reasonably adjusting the size of upload parts and the number of concurrent upload threads according to the machine's performance, better upload speeds can be achieved. General users do not need to make adjustments and can keep the default values. Additionally, there is an option to limit the maximum number of connections. If you do not want to limit the maximum number of connections, you can enter 0, which means there is no restriction on the maximum number of connections (however, you should evaluate this based on your machine's performance).
In the masquerade_address configuration item of the configuration file, it is generally recommended to specify the IP address used by the client to connect to the COS FTP Server. If you have any questions, please refer to the FTP Server Tool Frequently Asked Questions document.
Assume that the FTP Server has more than one IP address, and after running the ifconfig command, you get a private ENI IP 10.xxx.xxx.xxx, which is mapped to the public IP 119.xxx.xxx.xxx. At this time, if the FTP Server does not explicitly set masquerade_address to the public IP (119.xxx.xxx.xxx) that the client uses to access the server, the FTP Server in Passive mode may use the private IP (10.xxx.xxx.xxx) to return packets to the client. As a result, the client is able to connect to the FTP Server, but cannot return data packets to the client properly. Therefore, generally speaking, we recommend you to set masquerade_address to the IP address that your client is using to connect to the Server.
In the configuration file, the listen_port option is the listening port for the COS FTP Server, with a default value of 2121. The passive_port option is the data channel listening port range for the COS FTP Server, which defaults to a range between 60000 and 65535. When connecting to the COS FTP Server, ensure that the firewall allows traffic through the ports specified in listen_port and passive_port.

Quick Practice

Accessing COS FTP Server using Linux ftp command

1. Install the Linux FTP client.
yum install -y ftp
2. Open the Linux command line, and use the command ftp [ip address] [port No.] to connect to the COS FTP Server. Example:
ftp 192.xxx.xx.103 2121
In the FTP command, the IP setting corresponds to the masquerade_address configuration item in the example configuration file conf/vsftpd.conf.example. In this case, the IP is set to 192.xxx.xx.103.
In the FTP command, the port setting corresponds to the listen_port configuration item in the example configuration file conf/vsftpd.conf.example. In this case, it is set to 2121.
3. After running the above command, you will see Name and Password input fields. Enter the values configured in the COS FTP Server configuration items ftp_login_user_name and ftp_login_user_password to establish a successful connection.
Name: Corresponds to the ftp_login_user_name configuration item in the example configuration file conf/vsftpd.conf.example (requires configuration).
Password: Corresponds to the ftp_login_user_password configuration item in the example configuration file conf/vsftpd.conf.example (requires configuration).

Accessing COS FTP Server using FileZilla

1. Download and install FileZilla client.
2. After configuring the access information for COS FTP Server on your FileZilla client, click Quick Connect.
Host (H): Corresponds to the masquerade_address configuration item in the example configuration file conf/vsftpd.conf.example. In this example, the IP is set to 192.xxx.xx.103.
Note
If the COS FTP Server is behind a gateway or NAT, you can use this section to specify the gateway's IP address or domain name as the Server’s IP address.
Username (U): Corresponds to the ftp_login_user_name configuration item in the example configuration file conf/vsftpd.conf.example (requires configuration).
Password (W): Corresponds to the ftp_login_user_password configuration item in the example configuration file conf/vsftpd.conf.example (requires configuration).
Port (P): Corresponds to the listen_port configuration item in the example configuration file conf/vsftpd.conf.example. In this case, it is set to 2121.

FAQs

If any error occurs or you have any question on the upload limit while using FTP Server, see FTP Server FAQs.
中国站 - English