The content of this page has been automatically translated by AI. If you encounter any problems while reading, you can view the corresponding content in Chinese.

Table Definition in ProtoBuf

Last updated: 2024-10-15 17:13:56

TcaplusDB supports two table definition formats: Protobuf (Protocol Buffers, PB) and Tencent Data Representation (TDR). Both formats are equally effective. Choose the table definition language based on your specific business usage habits.
Protobuf is a descriptive language developed by Google for serializing structured data, emphasizing simplicity and performance.
TDR is a cross-platform data representation language developed by Tencent, combining the advantages of XML, binary, and ORM (Object-Relational Mapping), and is widely used in the serialization of game data by Tencent.
This document primarily describes the table definition format of Protobuf.

Protobuf Table

To create a table in TcaplusDB, you first need to use a table description language to define the table format. Write the table definition content into a file, which is called the table's IDL description file. The IDL description file of a Protobuf table is defined according to the rules of Protocol Buffers.
The main categories of definition are:
File Definition Information
Table Definition
Nested Type Information

File Definition Information

The File Definition Information section principally defines the public information for the current table description file, primarily involving three types of content:
Option Name
Feature Description
Sample Value
Required
syntax
Indicates the syntax specification version of the current file.
Supports proto2, proto3
Yes
package
Specify the current file's package name from the Definition. The package name can avoid name conflicts between message types.
Package Name Information
Yes
import
Introduce some public information of the Tcaplus table, which must be referenced in your table Definition.
tcaplusservice.optionv1.proto
Yes

Table Definition

The table Definition information is mainly formatted through message Definition tables. In the message, you can define extended information of the table or define the field information of the table.

Extended Information Definition

TcaplusDB table definitions can be extended through options based on protobuf syntax to achieve richer semantic features. The definable content is shown in the table below. The detailed definition format is:option(tcaplusservice.option) = "value";
Option Name
Feature Description
Configuration Example
Required
tcaplus_primary_key
Set the primary key field for the TcaplusDB table
option(tcaplusservice.tcaplus_primary_key) = "uin,name,region";
Yes
tcaplus_index
Set the index key field for the TcaplusDB table
option(tcaplusservice.tcaplus_index) = "index_1(uin,region)";
No
tcaplus_sharding_key
Users can define shard keys for the table
option(tcaplusservice.tcaplus_sharding_key) = "uin";
No
tcaplus_field_cipher_suite
If you need to use the field encryption feature, please set it using the example; if users need to specify their own encryption algorithm, refer to the example in the API
option(tcaplusservice.tcaplus_field_cipher_suite) = "DefaultAesCipherSuite";
No
tcaplus_cipher_md5
If you need to use the field encryption feature, you need to set the MD5 of the password string saved on the user side
option(tcaplusservice.tcaplus_cipher_md5)= "62fee3b53619b7f303c939964c6f2c4b";
No

Field Information Definition

The format of TcaplusDB definition fields is:Field modifier Field type Field name = Identification Number [special Definition];

Field Modifier

Proto2 supports three types of qualifying modifiers, while proto3 no longer supports the 'required' modifier and defaults to 'optional' type.
required: Indicates that the field is mandatory. In proto2, primary key fields must be marked as required.
optional: Indicates that the field is optional and supports setting a default value.
repeated: Indicates that the field can contain 0 - N elements. Its characteristics are similar to optional, but it can contain multiple values at a time, essentially passing an array of values. Must specify the special definition [packed = true].

Field Type

TcaplusDB supports both standard fields and nested fields. For a detailed introduction to fields, refer to Field Types.

Field Name

Name fields according to the current attributes. Supports uppercase and lowercase letters, numbers, and underscores. It is recommended to use camelCase for field naming, and it cannot start with a number.

Identifier

Range of identifier usage: [1, 2^29 - 1], cannot use identifiers [19000-19999] because these have been reserved in the Protobuf protocol implementation. Using these will result in an error. Each field occupies memory during encoding, and the amount of occupied memory depends on the identifier:
Fields with identifiers in the range [1, 15] occupy 1 byte during encoding.
Fields with identifiers in the range [16, 2047] occupy 2 bytes during encoding.

Special Definition

When a repeated field needs to specify the packed=true option. Usage is as follows:
repeated int64 lockid = 6 [packed = true];
Fields marked as optional can have a default value specified using default = 1. Usage is as follows:
optional int32 logintime = 5 [default = 1];
Fields of type string and bytes can be designated as encrypted fields. Usage is as follows:
required string name = 2[(tcaplusservice.tcaplus_crypto) = true];

Nested Type Information

TcaplusDB supports nested types. A nested type can contain another nested type as its field, and a new nested type can also be defined within a nested type. The definition of a nested type is similar to that of a table; both are declared using message, but the structure cannot include extended information definitions like "primary key" and "index". TcaplusDB supports up to 128 levels of continuous nesting, but heavy use of nesting is not recommended as too many nesting levels can degrade data access performance.

proto2 Table Description File Example

Here is a table description file that complies with the proto2 syntax specification:
syntax = "proto2"; // Indicates compliance with proto2 syntax specification
package myTcaplusTable; // Self-defined package name

import "tcaplusservice.optionv1.proto"; // This file defines some common information for Tcaplus tables and needs to be referenced (imported) in your table definition

message player { // Use message to define the table, the message name is the table name

// Only a message with the primary key option (tcaplusservice.tcaplus_primary_key) specified can be recognized as a TcaplusDB business data table; otherwise, this message is just a structure
// The primary key option specifies the list of primary key field names, separated by commas, with up to four fields being specified as primary keys
option(tcaplusservice.tcaplus_primary_key) = "uin,name,region";

// The tcaplusservice.tcaplus_index option specifies Tcaplus indexes
// Each index must include primary key fields, and the intersection of all index field sets must not be empty
option(tcaplusservice.tcaplus_index) = "index_1(uin,region)";
option(tcaplusservice.tcaplus_index) = "index_2(uin,name)";

// option(tcaplusservice.tcaplus_sharding_key) = "uin"; Users can explicitly set the index sharding field. If not explicitly set by the user, the system will default to using the primary key fields as sharding fields

option(tcaplusservice.tcaplus_field_cipher_suite) = "DefaultAesCipherSuite"; // Use the default DefaultAesCipherSuite encryption function, optional setting
option(tcaplusservice.tcaplus_cipher_md5)= "62fee3b53619b7f303c939964c6f2c4b"; // Set the MD5 value for the encrypted password string, optional setting

// Next is the table's Field Definition
// TcaplusDB table supports the following data types:
// Non-nested Type: int32, int64, uint32, uint64, sint32, sint64, bool, fixed64, sfixed64, double, fixed32, sfixed32, float, string, bytes
// Nested Type: message
// Tcaplus supports REQUIRED, OPTIONAL and REPEATED as Field Modifiers

// Primary Key Fields (up to 4)
required int64 uin = 1; // Primary key fields must be marked as required and cannot be of nested type
required string name = 2 [(tcaplusservice.tcaplus_crypto) = true]; // string and bytes fields in messages can be designated as encrypted fields, optional setting
required int32 region = 3;
// A table can contain up to 4 primary key fields

// Ordinary Value Field
required int32 gamesvrid = 4; // Ordinary fields can be marked as required, optional, or repeated
optional int32 logintime = 5 [default = 1];
repeated int64 lockid = 6 [packed = true]; // Fields marked as repeated need to have the packed=true option specified
optional bool is_available = 7 [default = false]; // Optional fields can specify default values
optional pay_info pay = 8; // The type of value field can be a self-defined structure type
}

message pay_info { // Using message definition structure

required int64 pay_id = 1;
optional uint64 total_money = 2;
optional uint64 pay_times = 3;
optional pay_auth_info auth = 4;

message pay_auth_info { // Structure type supports nested definition
required string pay_keys = 1;
optional int64 update_time = 2;
}
}

proto3 table description file example

The following is an example of a table description file that conforms to the proto3 syntax specification:
syntax = "proto3"; // Indicates compliance with the proto3 syntax specification
package myTcaplusTable; // Self-defined package name

import "tcaplusservice.optionv1.proto"; // This file defines some common information for Tcaplus tables and needs to be referenced (imported) in your table definition

message player { // Use message to define the table, the message name is the table name

// Only a message with the primary key option (tcaplusservice.tcaplus_primary_key) specified can be recognized as a TcaplusDB business data table; otherwise, this message is just a structure
// The primary key option specifies the list of primary key field names, separated by commas, with up to four fields being specified as primary keys
option(tcaplusservice.tcaplus_primary_key) = "uin,name,region";

// The tcaplusservice.tcaplus_index option specifies Tcaplus indexes
// Each index must include primary key fields, and the intersection of all index field sets must not be empty
option(tcaplusservice.tcaplus_index) = "index_1(uin,region)";
option(tcaplusservice.tcaplus_index) = "index_2(uin,name)";

// option(tcaplusservice.tcaplus_sharding_key) = "uin"; Users can explicitly set the sharding field themselves. If the user does not explicitly set it, the system will default to using the primary key field as the sharding field

option(tcaplusservice.tcaplus_field_cipher_suite) = "DefaultAesCipherSuite"; // Use the default DefaultAesCipherSuite encryption function, optional setting
option(tcaplusservice.tcaplus_cipher_md5)= "62fee3b53619b7f303c939964c6f2c4b"; // Set the MD5 value for the encrypted password string, optional setting

// Next is the table's Field Definition
// TcaplusDB table supports the following data types:
// Non-nested Type: int32, int64, uint32, uint64, sint32, sint64, bool, fixed64, sfixed64, double, fixed32, sfixed32, float, string, bytes
// Nested Type: message

// Primary Key Fields (up to 4)
int64 uin = 1; // Primary key fields must be non-nested types
string name = 2[(tcaplusservice.tcaplus_crypto) = true]; // String and bytes fields in messages can be designated as encrypted fields, optional setting
int32 region = 3;
// A table can contain up to 4 primary key fields

// Ordinary Value Field
int32 gamesvrid = 4;
int32 logintime = 5;
int64 lockid = 6;
bool is_available = 7;
pay_info pay = 8; // The type of value field can be a self-defined structure type
}

message pay_info { // Using message definition structure

int64 pay_id = 1;
uint64 total_money = 2;
uint64 pay_times = 3;
pay_auth_info auth = 4;

message pay_auth_info { // Structure type supports nested definition
string pay_keys = 1;
int64 update_time = 2;
}
}