前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >性能工具之 Goreplay 安装及入门使用

性能工具之 Goreplay 安装及入门使用

作者头像
高楼Zee
发布2021-09-02 15:12:28
2.5K0
发布2021-09-02 15:12:28
举报
文章被收录于专栏:7DGroup7DGroup

Goreplay 介绍

  • Goreplay 是用 Golang 写的一个 HTTP 实时流量复制工具。功能更强大,支持流量的放大、缩小,频率限制,还支持把请求记录到文件,方便回放和分析,也支持和 ElasticSearch 集成,将流量存入 ES 进行实时分析。
  • GoReplay 不是代理,而是监听网络接口上的流量,不需要更改生产基础架构,而是在与服务相同的计算机上运行 GoReplay 守护程序。
  • 特点:简单易用
  • 地址:https://github.com/buger/goreplay
  • stars:14.1k

与 TCPCopy 相比它的架构更简单,只有一个 gor 组件,如下:

只需要在生产服务器上启动一个 gor 进程,它负责所有的工作包括监听、过滤和转发。它的设计遵循 Unix 设计哲学:一切都是由管道组成的,各种输入将数据复用为输出。

Golang环境安装

一 、首先安装Golang及相关依赖环境 在官网上下载安装包或者go的一些中文网站上下载安装包

  • https://golang.org/dl/
  • https://studygolang.com/dl

我下载的是: go1.15.5.linux-amd64.tar.gz

二、解压到/usr/local目录下

代码语言:javascript
复制
tar -C /usr/local -zxvf go1.14.4.linux-amd64.tar.gz

三、配置环境变量

代码语言:javascript
复制
# 打开
vim /etc/profile

# 添加
export GOROOT=/usr/local/go
export PATH=$PATH:$GOROOT/bin

# 编译生效
source /etc/profile

四、验证 go env

代码语言:javascript
复制
[root@vm ~]# go version
go version go1.15.5 linux/amd64

Goreplay 安装

从https://github.com/buger/gor/releases 下载最新的 Gor 二进制文件(提供 Windows、Linux x64 和 Mac OS 的预编译二进制文件),也可以自己编译 。

下载二进制包:

代码语言:javascript
复制
[root@vm ~]# curl -L -O https://github.com/buger/goreplay/releases/download/v1.3.1/gor_1.3.1_x64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   626  100   626    0     0    741      0 --:--:-- --:--:-- --:--:--   741
100 10.5M  100 10.5M    0     0  3258k      0  0:00:03  0:00:03 --:--:-- 5951k

[root@vm-jmeter ~]# tar xvzf gor_1.3.1_x64.tar.gz 
gor

解压缩包后,您可以从当前目录运行 Gor,或者您可能希望将二进制文件复制到您的 PATH(对于 Linux 和 Mac OS,它可以是 /usr/local/bin)。

Goreplay 使用示例

准备 RESTful API 环境

接口设计:

请求类型

请求路径

功能

Get

/person

查询所有人

Post

/person/two

事务测试

打包程序分别放到两台服务器 run 起来。

服务一:

代码语言:javascript
复制
[root@vm-1 ~]# java -jar spirng-boot-demo-0.0.1-SNAPSHOT.jar 

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.3.8.RELEASE)

2021-08-17 14:19:10.467  WARN 17537 --- [           main] o.s.boot.StartupInfoLogger               : InetAddress.getLocalHost().getHostName() took 10014 milliseconds to respond. Please verify your network configuration.
2021-08-17 14:19:20.494  INFO 17537 --- [           main] com.zuozewei.SpirngbootdemoApplication   : Starting SpirngbootdemoApplication v0.0.1-SNAPSHOT on vm-jmeter with PID 17537 (/root/spirng-boot-demo-0.0.1-SNAPSHOT.jar started by root in /root)
2021-08-17 14:19:20.495  INFO 17537 --- [           main] com.zuozewei.SpirngbootdemoApplication   : The following profiles are active: a
2021-08-17 14:19:22.225  INFO 17537 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2021-08-17 14:19:22.355  INFO 17537 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 111ms. Found 1 JPA repository interfaces.
2021-08-17 14:19:23.642  INFO 17537 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8082 (http)
2021-08-17 14:19:23.667  INFO 17537 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-08-17 14:19:23.667  INFO 17537 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.41]
2021-08-17 14:19:23.804  INFO 17537 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2021-08-17 14:19:23.805  INFO 17537 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 3179 ms
2021-08-17 14:19:24.314  INFO 17537 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2021-08-17 14:19:24.479  INFO 17537 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate ORM core version 5.4.27.Final
2021-08-17 14:19:25.002  INFO 17537 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2021-08-17 14:19:25.228  INFO 17537 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2021-08-17 14:19:25.635  INFO 17537 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2021-08-17 14:19:25.706  INFO 17537 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.MySQL57Dialect
2021-08-17 14:19:27.086  INFO 17537 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2021-08-17 14:19:27.112  INFO 17537 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2021-08-17 14:19:27.990  WARN 17537 --- [           main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2021-08-17 14:19:28.261  INFO 17537 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2021-08-17 14:19:28.749  INFO 17537 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8082 (http) with context path ''
2021-08-17 14:19:28.770  INFO 17537 --- [           main] com.zuozewei.SpirngbootdemoApplication   : Started SpirngbootdemoApplication in 39.553 seconds (JVM running for 40.431)

服务二:

代码语言:javascript
复制
[root@vm-2 ~]# java -jar spirng-boot-demo-0.0.1-SNAPSHOT.jar 

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.3.8.RELEASE)

2021-08-17 14:18:34.814  INFO 12775 --- [           main] com.zuozewei.SpirngbootdemoApplication   : Starting SpirngbootdemoApplication v0.0.1-SNAPSHOT on vm-hsyjy-znkzb-vbi-1.novalocal with PID 12775 (/root/spirng-boot-demo-0.0.1-SNAPSHOT.jar started by root in /root)
2021-08-17 14:18:34.820  INFO 12775 --- [           main] com.zuozewei.SpirngbootdemoApplication   : The following profiles are active: a
2021-08-17 14:18:36.293  INFO 12775 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2021-08-17 14:18:36.419  INFO 12775 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 107ms. Found 1 JPA repository interfaces.
2021-08-17 14:18:37.694  INFO 12775 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8082 (http)
2021-08-17 14:18:37.716  INFO 12775 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-08-17 14:18:37.717  INFO 12775 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.41]
2021-08-17 14:18:37.841  INFO 12775 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2021-08-17 14:18:37.841  INFO 12775 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 2886 ms
2021-08-17 14:18:38.305  INFO 12775 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [name: default]
2021-08-17 14:18:38.444  INFO 12775 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate ORM core version 5.4.27.Final
2021-08-17 14:18:38.864  INFO 12775 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2021-08-17 14:18:39.119  INFO 12775 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
2021-08-17 14:18:39.556  INFO 12775 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
2021-08-17 14:18:39.648  INFO 12775 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dialect: org.hibernate.dialect.MySQL57Dialect
2021-08-17 14:18:41.052  INFO 12775 --- [           main] o.h.e.t.j.p.i.JtaPlatformInitiator       : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2021-08-17 14:18:41.086  INFO 12775 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2021-08-17 14:18:42.042  WARN 12775 --- [           main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2021-08-17 14:18:42.330  INFO 12775 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2021-08-17 14:18:42.768  INFO 12775 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8082 (http) with context path ''
2021-08-17 14:18:42.786  INFO 12775 --- [           main] com.zuozewei.SpirngbootdemoApplication   : Started SpirngbootdemoApplication in 8.988 seconds (JVM running for 9.939)

捕获服务一流量保存到本地文件

在终端运行这个命令:

代码语言:javascript
复制
#将端口 8082 流量保存到本地的文件
sudo ./gor --input-raw :8082 --output-file=requests.gor
代码语言:javascript
复制
[root@vm-1 ~]# sudo ./gor --input-raw :8082 --output-file=requests.gor
Interface: eth0 . BPF Filter: ((tcp dst port 8082) and (dst host 172.16.106.149 or dst host fe80::f599:6e65:3dc9:b84a))
Interface: lo . BPF Filter: ((tcp dst port 8082) and (dst host 127.0.0.1 or dst host ::1))
Interface: virbr0 . BPF Filter: ((tcp dst port 8082) and (dst host 192.168.124.1))
2021/08/17 14:26:41 [PPID 17654 and PID 17655] Version:1.3.0

注意:使用--output-file-append 选项可以将捕获流量保存为一个单独的文件 使用sudo并要求输入密码:为了分析网络,Gor 需要只有超级用户才能使用的权限。但是,可以为非 root 用户配置运行Gor 。

在 Postman 上向服务一发送请求:

我们查看服务一日志:

代码语言:javascript
复制
Hibernate: select person0_.id as id1_0_, person0_.age as age2_0_, person0_.name as name3_0_ from person person0_
Hibernate: select next_val as id_val from hibernate_sequence for update
Hibernate: update hibernate_sequence set next_val= ? where next_val=?
Hibernate: select next_val as id_val from hibernate_sequence for update
Hibernate: update hibernate_sequence set next_val= ? where next_val=?
Hibernate: insert into person (age, name, id) values (?, ?, ?)
Hibernate: insert into person (age, name, id) values (?, ?, ?)

可以看到服务一已经正常处理了。

我们停掉在服务器一上运行的 Gor,可以看到已经生成了录制的请求文件:

代码语言:javascript
复制
[root@vm-1 ~]# ls -l | grep "requests"
-rw-r----- 1 root    root         300 8月  17 14:33 requests_0.gor

我们看下请求文件的内容:

代码语言:javascript
复制
[root@vm-1 ~]# cat requests_0.gor 
1 04481f92ac106afb42ce55ca 1629182282437724065 0
GET /person HTTP/1.1
User-Agent: PostmanRuntime/7.28.2
Accept: */*
Cache-Control: no-cache
Postman-Token: ab405d20-ef93-4ffb-b2f6-a16cf1205e7c
Host: 172.16.106.149:8082
Accept-Encoding: gzip, deflate, br
Connection: keep-alive

将流量从文件回放到服务二

现在是时候将原始流量回放到服务二中了。如果是多个请求并且它们将按照与记录的顺序和时间完全相同的顺序重放。

--output-http 提供第二个服务的 URL

代码语言:javascript
复制
./gor --input-file requests_0.gor --output-http="http://172.16.106.237:8082"
代码语言:javascript
复制
[root@vm-1 ~]./gor --input-file requests_0.gor --output-http="http://172.16.106.237:8082"
2021/08/17 14:42:55 [PPID 12356 and PID 17914] Version:1.3.0
[DEBUG][elapsed 985.792µs]: [INPUT-FILE] FileInput: end of file 'requests_0.gor'

我们看到服务二的日志:

代码语言:javascript
复制
[root@vm-2 ~]# java -jar spirng-boot-demo-0.0.1-SNAPSHOT.jar 

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.3.8.RELEASE)

2021-08-17 14:18:34.814  INFO 12775 --- [           main] com.zuozewei.SpirngbootdemoApplication   : Starting SpirngbootdemoApplication v0.0.1-SNAPSHOT on vm-hsyjy-znkzb-vbi-1.novalocal with PID 12775 (/root/spirng-boot-demo-0.0.1-SNAPSHOT.jar started by root in /root)
......

Hibernate: select person0_.id as id1_0_, person0_.age as age2_0_, person0_.name as name3_0_ from person person0_

日志说明服务二正常响应。

将 http 的请求打印到终端

通过命令:

代码语言:javascript
复制
sudo ./gor --input-raw :8082 --output-stdout

此命令其记录到标准控制台输出,请注意,默认情况下 GoReplay 不跟踪响应,您可以使用 --output-http-track-response 选项启用它们。

代码语言:javascript
复制
[root@vm-1 ~]sudo ./gor --input-raw :8082 --output-stdout
Interface: eth0 . BPF Filter: ((tcp dst port 8082) and (dst host 172.16.106.149 or dst host fe80::f599:6e65:3dc9:b84a))
Interface: lo . BPF Filter: ((tcp dst port 8082) and (dst host 127.0.0.1 or dst host ::1))
Interface: virbr0 . BPF Filter: ((tcp dst port 8082) and (dst host 192.168.124.1))
2021/08/17 14:55:45 [PPID 18060 and PID 18061] Version:1.3.0
1 22ee1f92ac106afb594d205a 1629183362175214036 0
POST /person/two HTTP/1.1
User-Agent: PostmanRuntime/7.28.2
Accept: */*
Cache-Control: no-cache
Postman-Token: 4922ce23-5706-42c5-8a54-ade5d5aee785
Host: 172.16.106.149:8082
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 0

将 http 的请求实时转发到服务二

运行命令:

代码语言:javascript
复制
sudo ./gor --input-raw :8082 --output-http="http://172.16.106.237:8082"
代码语言:javascript
复制
[root@vm-1 ~]# sudo ./gor --input-raw :8082 --output-http="http://172.16.106.237:8082"
Interface: eth0 . BPF Filter: ((tcp dst port 8082) and (dst host 172.16.106.149 or dst host fe80::f599:6e65:3dc9:b84a))
Interface: lo . BPF Filter: ((tcp dst port 8082) and (dst host 127.0.0.1 or dst host ::1))
Interface: virbr0 . BPF Filter: ((tcp dst port 8082) and (dst host 192.168.124.1))
2021/08/17 14:59:49 [PPID 18118 and PID 18119] Version:1.3.0

在服务器二查看日志,.发现流量已经进来了:

代码语言:javascript
复制
Hibernate: select person0_.id as id1_0_, person0_.age as age2_0_, person0_.name as name3_0_ from person person0_
Hibernate: select person0_.id as id1_0_, person0_.age as age2_0_, person0_.name as name3_0_ from person person0_
Hibernate: select person0_.id as id1_0_, person0_.age as age2_0_, person0_.name as name3_0_ from person person0_ where person0_.age=?

压力测试

GOR 支持将捕获到的生产实际请求流量减少或者放大回放以用于各种环境的压力测试。

录制的请求文件:

代码语言:javascript
复制
[root@vm-1 ~]# cat requests.gor 
1 040f1f92ac106afb7e8d5679 1629183084044026882 0
POST /person/two HTTP/1.1
User-Agent: PostmanRuntime/7.28.2
Accept: */*
Cache-Control: no-cache
Postman-Token: a8600dd1-28c1-4825-b6a2-68c42659942f
Host: 172.16.106.149:8082
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 0



[root@vm-jmeter ~]# cat requests.gor 
1 040f1f92ac106afb7e8d5679 1629183084044026882 0
POST /person/two HTTP/1.1
User-Agent: PostmanRuntime/7.28.2
Accept: */*
Cache-Control: no-cache
Postman-Token: a8600dd1-28c1-4825-b6a2-68c42659942f
Host: 172.16.106.149:8082
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 0

我们可以看到主要是两个请求。

运行以下命令,将流量从文件回放到服务二,并放大两倍:

代码语言:javascript
复制
[root@vm-1 ~]./gor --input-file "requests.gor|200%" --output-http="http://172.16.106.237:8082"
2021/08/17 15:03:58 [PPID 12356 and PID 18187] Version:1.3.0
[DEBUG][elapsed 1.361742ms]: [INPUT-FILE] FileInput: end of file 'requests.gor'

我们查看下服务二日志的处理情况:

代码语言:javascript
复制
Hibernate: select next_val as id_val from hibernate_sequence for update
Hibernate: update hibernate_sequence set next_val= ? where next_val=?
Hibernate: select next_val as id_val from hibernate_sequence for update
Hibernate: update hibernate_sequence set next_val= ? where next_val=?
Hibernate: insert into person (age, name, id) values (?, ?, ?)
Hibernate: insert into person (age, name, id) values (?, ?, ?)

注意,当然也也支持请求流量 10%,20% 等缩小。

总结

今天简单介绍 Gor 的常见玩法,事实上远不止上述提到的这些,更多的玩法有时间再写。

文件源码:

  • https://github.com/zuozewei/blog-example/tree/master/Performance-testing/04-full-link/springboot-demo
本文参与 腾讯云自媒体分享计划,分享自微信公众号。
原始发表:2021-08-17,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 7DGroup 微信公众号,前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体分享计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • Goreplay 介绍
  • Golang环境安装
  • Goreplay 安装
  • Goreplay 使用示例
    • 准备 RESTful API 环境
      • 捕获服务一流量保存到本地文件
        • 将流量从文件回放到服务二
          • 将 http 的请求打印到终端
            • 将 http 的请求实时转发到服务二
              • 压力测试
              • 总结
              相关产品与服务
              Serverless HTTP 服务
              Serverless HTTP 服务基于腾讯云 API 网关 和 Web Cloud Function(以下简称“Web Function”)建站云函数(云函数的一种类型)的产品能力,可以支持各种类型的 HTTP 服务开发,实现了 Serverless 与 Web 服务最优雅的结合。用户可以快速构建 Web 原生框架,把本地的 Express、Koa、Nextjs、Nuxtjs 等框架项目快速迁移到云端,同时也支持 Wordpress、Discuz Q 等现有应用模版一键快速创建。
              领券
              问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档