TcaplusDB supports two table definition formats: Protobuf (Protocol Buffers, PB) and Tencent Data Representation (TDR).
Both formats are equally effective. Choose the table definition language based on your specific business usage habits.
Protobuf is a descriptive language developed by Google for serializing structured data, emphasizing simplicity and performance.
TDR is a cross-platform data representation language developed by Tencent, combining the advantages of XML, binary, and ORM (Object-Relational Mapping), and is widely used in the serialization of game data by Tencent.
This document primarily describes the table definition format of Protobuf.
Protobuf Table
To create a table in TcaplusDB, you first need to use a table description language to define the table format. Write the table definition content into a file, which is called the table's IDL description file.
The IDL description file of a Protobuf table is defined according to the rules of Protocol Buffers.
The main categories of definition are:
File Definition Information
Table Definition
Nested Type Information
File Definition Information
The File Definition Information section principally defines the public information for the current table description file, primarily involving three types of content:
Option Name | Feature Description | Sample Value | Required |
syntax | Indicates the syntax specification version of the current file. | Supports proto2, proto3 | Yes |
package | Specify the current file's package name from the Definition. The package name can avoid name conflicts between message types. | Package Name Information | Yes |
import | Introduce some public information of the Tcaplus table, which must be referenced in your table Definition. | tcaplusservice.optionv1.proto | Yes |
Table Definition
The table Definition information is mainly formatted through message Definition tables. In the message, you can define extended information of the table or define the field information of the table.
Extended Information Definition
TcaplusDB table definitions can be extended through options based on protobuf syntax to achieve richer semantic features. The definable content is shown in the table below.
The detailed definition format is:
option(tcaplusservice.option) = "value";Option Name | Feature Description | Configuration Example | Required |
tcaplus_primary_key | Set the primary key field for the TcaplusDB table | option(tcaplusservice.tcaplus_primary_key) = "uin,name,region"; | Yes |
tcaplus_index | Set the index key field for the TcaplusDB table | option(tcaplusservice.tcaplus_index) = "index_1(uin,region)"; | No |
tcaplus_sharding_key | Users can define shard keys for the table | option(tcaplusservice.tcaplus_sharding_key) = "uin"; | No |
tcaplus_field_cipher_suite | If you need to use the field encryption feature, please set it using the example; if users need to specify their own encryption algorithm, refer to the example in the API | option(tcaplusservice.tcaplus_field_cipher_suite) = "DefaultAesCipherSuite"; | No |
tcaplus_cipher_md5 | If you need to use the field encryption feature, you need to set the MD5 of the password string saved on the user side | option(tcaplusservice.tcaplus_cipher_md5)= "62fee3b53619b7f303c939964c6f2c4b"; | No |
Field Information Definition
The format of TcaplusDB definition fields is:
Field modifier Field type Field name = Identification Number [special Definition];Field Modifier
Proto2 supports three types of qualifying modifiers, while proto3 no longer supports the 'required' modifier and defaults to 'optional' type.
required: Indicates that the field is mandatory. In proto2, primary key fields must be marked as required.
optional: Indicates that the field is optional and supports setting a default value.
repeated: Indicates that the field can contain 0 - N elements. Its characteristics are similar to optional, but it can contain multiple values at a time, essentially passing an array of values. Must specify the special definition [packed = true].
Field Type
TcaplusDB supports both standard fields and nested fields. For a detailed introduction to fields, refer to Field Types.
Field Name
Name fields according to the current attributes. Supports uppercase and lowercase letters, numbers, and underscores. It is recommended to use camelCase for field naming, and it cannot start with a number.
Identifier
Range of identifier usage: [1, 2^29 - 1], cannot use identifiers [19000-19999] because these have been reserved in the Protobuf protocol implementation. Using these will result in an error.
Each field occupies memory during encoding, and the amount of occupied memory depends on the identifier:
Fields with identifiers in the range [1, 15] occupy 1 byte during encoding.
Fields with identifiers in the range [16, 2047] occupy 2 bytes during encoding.
Special Definition
When a repeated field needs to specify the packed=true option. Usage is as follows:
repeated int64 lockid = 6 [packed = true];
Fields marked as optional can have a default value specified using default = 1. Usage is as follows:
optional int32 logintime = 5 [default = 1];
Fields of type string and bytes can be designated as encrypted fields. Usage is as follows:
required string name = 2[(tcaplusservice.tcaplus_crypto) = true];
Nested Type Information
TcaplusDB supports nested types. A nested type can contain another nested type as its field, and a new nested type can also be defined within a nested type.
The definition of a nested type is similar to that of a table; both are declared using message, but the structure cannot include extended information definitions like "primary key" and "index".
TcaplusDB supports up to 128 levels of continuous nesting, but heavy use of nesting is not recommended as too many nesting levels can degrade data access performance.
proto2 Table Description File Example
Here is a table description file that complies with the proto2 syntax specification:
syntax = "proto2"; // Indicates compliance with proto2 syntax specificationpackage myTcaplusTable; // Self-defined package nameimport "tcaplusservice.optionv1.proto"; // This file defines some common information for Tcaplus tables and needs to be referenced (imported) in your table definitionmessage player { // Use message to define the table, the message name is the table name// Only a message with the primary key option (tcaplusservice.tcaplus_primary_key) specified can be recognized as a TcaplusDB business data table; otherwise, this message is just a structure// The primary key option specifies the list of primary key field names, separated by commas, with up to four fields being specified as primary keysoption(tcaplusservice.tcaplus_primary_key) = "uin,name,region";// The tcaplusservice.tcaplus_index option specifies Tcaplus indexes// Each index must include primary key fields, and the intersection of all index field sets must not be emptyoption(tcaplusservice.tcaplus_index) = "index_1(uin,region)";option(tcaplusservice.tcaplus_index) = "index_2(uin,name)";// option(tcaplusservice.tcaplus_sharding_key) = "uin"; Users can explicitly set the index sharding field. If not explicitly set by the user, the system will default to using the primary key fields as sharding fieldsoption(tcaplusservice.tcaplus_field_cipher_suite) = "DefaultAesCipherSuite"; // Use the default DefaultAesCipherSuite encryption function, optional settingoption(tcaplusservice.tcaplus_cipher_md5)= "62fee3b53619b7f303c939964c6f2c4b"; // Set the MD5 value for the encrypted password string, optional setting// Next is the table's Field Definition// TcaplusDB table supports the following data types:// Non-nested Type: int32, int64, uint32, uint64, sint32, sint64, bool, fixed64, sfixed64, double, fixed32, sfixed32, float, string, bytes// Nested Type: message// Tcaplus supports REQUIRED, OPTIONAL and REPEATED as Field Modifiers// Primary Key Fields (up to 4)required int64 uin = 1; // Primary key fields must be marked as required and cannot be of nested typerequired string name = 2 [(tcaplusservice.tcaplus_crypto) = true]; // string and bytes fields in messages can be designated as encrypted fields, optional settingrequired int32 region = 3;// A table can contain up to 4 primary key fields// Ordinary Value Fieldrequired int32 gamesvrid = 4; // Ordinary fields can be marked as required, optional, or repeatedoptional int32 logintime = 5 [default = 1];repeated int64 lockid = 6 [packed = true]; // Fields marked as repeated need to have the packed=true option specifiedoptional bool is_available = 7 [default = false]; // Optional fields can specify default valuesoptional pay_info pay = 8; // The type of value field can be a self-defined structure type}message pay_info { // Using message definition structurerequired int64 pay_id = 1;optional uint64 total_money = 2;optional uint64 pay_times = 3;optional pay_auth_info auth = 4;message pay_auth_info { // Structure type supports nested definitionrequired string pay_keys = 1;optional int64 update_time = 2;}}
proto3 table description file example
The following is an example of a table description file that conforms to the proto3 syntax specification:
syntax = "proto3"; // Indicates compliance with the proto3 syntax specificationpackage myTcaplusTable; // Self-defined package nameimport "tcaplusservice.optionv1.proto"; // This file defines some common information for Tcaplus tables and needs to be referenced (imported) in your table definitionmessage player { // Use message to define the table, the message name is the table name// Only a message with the primary key option (tcaplusservice.tcaplus_primary_key) specified can be recognized as a TcaplusDB business data table; otherwise, this message is just a structure// The primary key option specifies the list of primary key field names, separated by commas, with up to four fields being specified as primary keysoption(tcaplusservice.tcaplus_primary_key) = "uin,name,region";// The tcaplusservice.tcaplus_index option specifies Tcaplus indexes// Each index must include primary key fields, and the intersection of all index field sets must not be emptyoption(tcaplusservice.tcaplus_index) = "index_1(uin,region)";option(tcaplusservice.tcaplus_index) = "index_2(uin,name)";// option(tcaplusservice.tcaplus_sharding_key) = "uin"; Users can explicitly set the sharding field themselves. If the user does not explicitly set it, the system will default to using the primary key field as the sharding fieldoption(tcaplusservice.tcaplus_field_cipher_suite) = "DefaultAesCipherSuite"; // Use the default DefaultAesCipherSuite encryption function, optional settingoption(tcaplusservice.tcaplus_cipher_md5)= "62fee3b53619b7f303c939964c6f2c4b"; // Set the MD5 value for the encrypted password string, optional setting// Next is the table's Field Definition// TcaplusDB table supports the following data types:// Non-nested Type: int32, int64, uint32, uint64, sint32, sint64, bool, fixed64, sfixed64, double, fixed32, sfixed32, float, string, bytes// Nested Type: message// Primary Key Fields (up to 4)int64 uin = 1; // Primary key fields must be non-nested typesstring name = 2[(tcaplusservice.tcaplus_crypto) = true]; // String and bytes fields in messages can be designated as encrypted fields, optional settingint32 region = 3;// A table can contain up to 4 primary key fields// Ordinary Value Fieldint32 gamesvrid = 4;int32 logintime = 5;int64 lockid = 6;bool is_available = 7;pay_info pay = 8; // The type of value field can be a self-defined structure type}message pay_info { // Using message definition structureint64 pay_id = 1;uint64 total_money = 2;uint64 pay_times = 3;pay_auth_info auth = 4;message pay_auth_info { // Structure type supports nested definitionstring pay_keys = 1;int64 update_time = 2;}}