Test Tool
The open-source script of the Kafka client can be used for Kafka producer and consumer performance testing. Test results are displayed mainly based on the size of messages sent per second (MB/second) and the number of messages sent per second (records/second).
Kafka Producer Test Script:
$KAFKA_HOME/bin/kafka-producer-perf-test.shKafka Consumer Test Script:
$KAFKA_HOME/bin/kafka-consumer-perf-test.shTest Command
Note
In the following command, replace
ckafka vip:vport with the actual IP and port assigned to your instance.Sample command for production testing:
bin/kafka-producer-perf-test.sh--topic test--num-records 123--record-size 1000--producer-props bootstrap.servers= ckafka vip : port--throughput 20000
Sample command for consumption testing:
bin/kafka-consumer-perf-test.sh--topic test--new-consumer--fetch-size 10000--messages 1000--broker-list bootstrap.servers=ckafka vip : port
Testing Recommendations
We recommend you create three or more partitions to increase throughput. This is because there must be at least three CKafka cluster nodes at the backend. If only one partition is created, it will be distributed in a single broker, which will affect CKafka performance.
As messages in each CKafka partition are ordered, the production performance will be affected if there are too many partitions. We recommend you create up to six partitions.
To ensure the effectiveness of the stress test, it is necessary to simulate a certain level of concurrency with multiple clients. It is recommended to use multiple machines as stress test clients (producers), with each machine running multiple test programs to increase concurrency. Additionally, it is advised to start a producer every 1 second to avoid high load on the test machine caused by starting all producers simultaneously.