前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >测试常用curl工具详解

测试常用curl工具详解

原创
作者头像
sheazhang
修改2019-09-02 11:25:30
3.7K0
修改2019-09-02 11:25:30
举报
文章被收录于专栏:加速视频

前言

linux中测试网络的访问工具

代码语言:javascript
复制
title:      CDN中测试常用curl工具详解
author:     sheazhang

1. 验证某节点ip缓存情况

代码语言:javascript
复制
<p style="line-height: 2em;">[root@VM_27_201_centos ~]# curl -I 'http://app613.imgcache.xxx.com/app613/frontend_swf/version.js' -x 106.74.23.144:80
HTTP/1.1 200 OK
Server: NWS_TCloud_S1
Connection: keep-alive
Date: Sun, 03 Jul 2016 08:50:34 GMT
Cache-Control: max-age=6000
Expires: Sun, 03 Jul 2016 10:30:34 GMT
Last-Modified: Thu, 30 Jun 2016 09:12:30 GMT
Content-Type: application/x-javascript; charset=utf-8
Content-Length: 69
X-Cache-Lookup: Hit From Disktank<br></p>

PS: 其中的-x绑定代理服务器ip的用法,类似于wget中的:

代码语言:javascript
复制
# wget -e http-proxy=119.97.137.178 http://sdxxx.cloudcdn.net/test008/6wireshark-win64-1.2.9.exe

get back

2. 验证referer头部白名单是否生效

场景

使用-H 来添加自定义的头部,模拟特殊条件下源站/cdn节点对特性是否支持,结果返回是否符合预期。下面的这个例子,就是用来验证在控制台上添加了referer白名单后,模拟该referer头部返回是否符合预期。

代码语言:javascript
复制
<p style="line-height: 2em;">[root@VM_27_201_centos ~]# curl -I 'http://res2.xxxxx.com/' -x 106.74.23.144:80 -H"referer:http://*.yqh5.cn"
HTTP/1.1 200 OK
Server: NWS_TCloud_S1
Connection: keep-alive
Date: Sun, 03 Jul 2016 08:59:50 GMT
Cache-Control: max-age=31536000
Expires: Mon, 03 Jul 2017 08:59:50 GMT
Last-Modified: Tue, 12 Apr 2016 01:59:32 GMT
Content-Type: text/html
Content-Length: 612
X-Cache-Lookup: Hit From Disktank
Access-Control-Allow-Origin: *
Content-Disposition: inline; filename=""
Accept-Ranges: bytes<br></p>

PS 上面的

代码语言:javascript
复制
curl -I 'http://res2.eqxxx.com/' -x 106.74.23.144:80 -H"referer:http://*.yqh5.cn"

也可以写为:

代码语言:javascript
复制
curl -I 'http://res2.exxx.com/' -x 106.74.23.144:80  -e "http://pachong.org/"  

get back

3. 验证节点/源站对分片的支持情况

代码语言:javascript
复制
[root@VM_27_201_centos ~]# curl -Iv 'http://update01.xxxx-cloud.com/pup/8S70_E6200_V016.002.230_9.zip' -x 106.74.23.144:80 -H'range:bytes=0-1000'
* About to connect() to proxy 106.74.23.144 port 80 (#0)
*   Trying 106.74.23.144... connected
* Connected to 106.74.23.144 (106.74.23.144) port 80 (#0)
> HEAD http://update01.skyworth-cloud.com/pup/8S70_E6200_V016.002.230_9.zip HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: update01.skyworth-cloud.com
> Accept: */*
> Proxy-Connection: Keep-Alive
> range:bytes=0-1000
>
< HTTP/1.1 206 Partial Content
HTTP/1.1 206 Partial Content
< Server: nws_ocmid_hy
Server: nws_ocmid_hy
< Connection: close
Connection: close
< Date: Sun, 03 Jul 2016 09:02:24 GMT
Date: Sun, 03 Jul 2016 09:02:24 GMT
< Cache-Control: max-age=600
Cache-Control: max-age=600
< Expires: Sun, 03 Jul 2016 09:12:24 GMT
Expires: Sun, 03 Jul 2016 09:12:24 GMT
< Last-Modified: Mon, 23 May 2016 20:10:06 GMT
Last-Modified: Mon, 23 May 2016 20:10:06 GMT
< Content-Range: bytes 0-1000/437405243
Content-Range: bytes 0-1000/437405243
< Content-Type: application/octet-stream
Content-Type: application/octet-stream
< Content-Length: 1001
Content-Length: 1001
< X-Daa-Tunnel: hop_count=2
X-Daa-Tunnel: hop_count=2
< X-Cache-Lookup: Hit From Disktank Upstream
X-Cache-Lookup: Hit From Disktank Upstream
< X-Cache-Lookup: Hit From Disktank
X-Cache-Lookup: Hit From Disktank
< X-Daa-Tunnel: hop_count=2
X-Daa-Tunnel: hop_count=2
< X-Cache-Lookup: Hit From Inner Cluster
X-Cache-Lookup: Hit From Inner Cluster
 
<
* Closing connection #0<br><br></p>

备注

上面的

代码语言:javascript
复制
curl -Iv 'http://update01.xxx-cloud.com/pup/8S70_E6200_V016.002.230_9.zip' -x 106.74.23.144:80 -H'range:bytes=0-1000'

也可以写成

代码语言:javascript
复制
curl -Iv 'http://update01.xxx-cloud.com/pup/8S70_E6200_V016.002.230_9.zip' -x 106.74.23.144:80 -r 0-1000

get back

4. 验证文件在节点和源站上大小是否一致(通过md5判断)

代码语言:javascript
复制
[root@VM_27_201_centos ~]# curl -o pic1 'http://as.xxx.com/show/visitor/4a2d8aae508506ef0150a6fcebbf2f53' -x 106.74.23.144:80
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10126  100 10126    0     0  40629      0 --:--:-- --:--:-- --:--:-- 54149
[root@VM_27_201_centos ~]#
[root@VM_27_201_centos ~]# md5sum pic1
4b7671dbe1a634541726bd9473d14dcc  pic1
[root@VM_27_201_centos ~]#
[root@VM_27_201_centos ~]#
[root@VM_27_201_centos ~]# curl -o pic2 'http://as.xxx.com/show/visitor/4a2d8aae508506ef0150a6fcebbf2f53' -x 115.159.20.55:80
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10126  100 10126    0     0  4629k      0 --:--:-- --:--:-- --:--:-- 9888k
[root@VM_27_201_centos ~]#
[root@VM_27_201_centos ~]# md5sum pic2
4b7671dbe1a634541726bd9473d14dcc  pic2<br></p>

备注

上面的-o 代表把文件下载到指定的路径下,-x代表使用指定的ip或者域名作为代理。

get back

5. 模拟https访问验证https证书

代码语言:javascript
复制
[root@VM_117_214_centos ~]#  curl -sv 'https://xxx.qcloud.com/static/img/57c45bb3997a277e5996dcf13314b365/user-40.png' -o /dev/null -k
* About to connect() to mccdn.qcloud.com port 443 (#0)
*   Trying 113.207.16.63... connected
* Connected to mccdn.qcloud.com (113.207.16.63) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
*     subject: CN=*.qcloud.com,OU=R&D,O=Shenzhen Tencent Computer Systems Company Limited,L=shenzhen,ST=guangdong,C=CN
*     start date: May 25 00:00:00 2016 GMT
*     expire date: Jul 24 23:59:59 2019 GMT
*     common name: *.qcloud.com
*     issuer: CN=Symantec Class 3 Secure Server CA - G4,OU=Symantec Trust Network,O=Symantec Corporation,C=US
> GET /static/img/57c45bb3997a277e5996dcf13314b365/user-40.png HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.21 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: mccdn.qcloud.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nnws/1.7.3.8
< Date: Sat, 20 Aug 2016 12:43:55 GMT
< Content-Type: image/png
< Content-Length: 3558
< Connection: keep-alive
< Cache-Control: max-age=2592000
< Expires: Mon, 19 Sep 2016 12:43:54 GMT
< Last-Modified: Wed, 25 May 2016 08:06:29 GMT
< X-Cache-Lookup: Hit From Disktank
< 
{ [data not shown]
* Connection #0 to host mccdn.qcloud.com left intact
* Closing connection #0

get back

6. curl中绑定host测试方法

等同于在windows中绑定host进行测试

代码语言:javascript
复制
[root@VM_230113centos ~]# curl -I -H 'host:7u2q8y.com3.z0.glb.xxxcdn.com' http://xxx.com.z0.glb.xxxdns.com/M-_WoLgVqduR7VFfrhnmWwL8YgA=/luI-
 
lideKDodDAYi55H6D2HV6HcO/000000.ts
HTTP/1.1 200 OK
Server: nginx/1.4.4
Date: Sat, 02 Jan 2016 11:39:23 GMT
Content-Type: video/mp2t
Content-Length: 673416
Connection: keep-alive
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 2592000
Cache-Control: public, max-age=31536000
Content-Disposition: inline; filename="000000.ts"
Content-Transfer-Encoding: binary
Etag: "Fs–ekImTQ2ZMjO_MdTeTN0Yew_5"
Last-Modified: Sun, 22 Nov 2015 06:28:55 GMT
X-Log: mc.g;IO:1
X-Reqid: iToAAMIdW1cTmSUU
X-Qiniu-Zone: 0

[root@VM_230113centos ~]# curl -I -H 'host:7u2q8y.com3.z0.glb.xxxcdn.com' http://211.161.127.147/M-_WoLgVqduR7VFfrhnmWwL8YgA=/luI-lideKDodDAYi55H6D2HV6HcO/000000.ts
HTTP/1.1 200 OK
Server: NWS_Appimg_HY
Connection: keep-alive
Date: Sat, 02 Jan 2016 11:45:50 GMT
Cache-Control: max-age=31536000
Expires: Sun, 01 Jan 2017 11:45:50 GMT
Last-Modified: Sun, 22 Nov 2015 06:28:55 GMT
Content-Type: video/mp2t
Content-Length: 673416
X-Cache-Lookup: Hit From Disktank
Access-Control-Allow-Origin: *
Accept-Ranges: bytes
Etag: "Fs–ekImTQ2ZMjO_MdTeTN0Yew_5"
X-ReqId: ATUAABGu8oBVjyUU
Content-Disposition: inline; filename="000000.ts" 

其中:

  • 7u2q8y.com3.z0.glb.qiniucdn.com 为用户的域名
  • src.com.z0.glb.qiniudns.com 为源站或者回源host地址,也可以为ip地址形式
  • 211.161.127.147 为域名接入cdn之后解析到的cdn节点ip地址

通过验证源站url和cdn节点ip地址后的curl反馈可以判断出到底是源站有问题,还是cdn节点有问题

get back

7. 验证TCP握手和ssl握手的延时

代码语言:javascript
复制
curl -w "TCP handshake: %{time_connect}, SSL handshake: %{time_appconnect}\n" -so /dev/null https://www.alipay.com

8. 使用curl抓取返回时间

判断网页访问各耗时时间

代码语言:javascript
复制
curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} \n" http://www.itts-union.com
This command will output this:
[root@VM_117_214_centos ~]#  curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} \n" http://www.itts-union.com/1326.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 74470    0 74470    0     0  52368      0 --:--:--  0:00:01 --:--:--  116k
Connect: 0.799 TTFB: 1.309 Total time: 1.422

get back

9. 查询各个阶段的耗时时间:

-w 可以指定特殊文件 也可以指定耗时**%{time_namelookup}**

代码语言:javascript
复制
[root@VM_49_231_centos ~]# curl -Iv "http://img.yxp01.com/
images/20/2017/07/VXunsmRXXU355lC5lM8xc5sXmFNfus.jpg?v=4" -x 
119.6.229.37:80 -so /dev/null -w "@curl-time"

特殊文件:

代码语言:javascript
复制
[root@VM_49_231_centos ~]# cat curl-time 
    Time-DNS-lookup:  %{time_namelookup}s\n
   Time-TCP-connect:  %{time_connect}s\n
   Time-SSL-connect:  %{time_appconnect}s\n
      Time-redirect:  %{time_redirect}s\n
  Time-Pre-transfer:  %{time_pretransfer}s\n
Time-First-transfer:  %{time_starttransfer}s\n
                    ----------\n
         Time-total:  %{time_total}s\n

get back

10. -L实现301或者302的跳转

11. 缓存文件建立时间

Pragma: X-Get-Last-Update-Info

get back

12. 查看cdn节点回源的链路

x_upstream_info=on

代码语言:javascript
复制
curl -o 100 "http://res.hoisin.xxxtv.com/video/20150731164143_960.ts?x_upstream_info=on" -x 61.140.13.201:80 -H "Pragma: X-Get-Last-Update-Info" -v

[root@sheazhang ~]# curl -o /dev/null -v "http://zl.winyp.cn/test.jpg?x_upstream_info=on"
> GET /test.jpg?x_upstream_info=on HTTP/1.1
> User-Agent: curl/7.29.0
> Host: zl.winyp.cn
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: NWS_TCloud_S1
< Connection: keep-alive
< Date: Tue, 01 Jan 2019 02:46:29 GMT
< Cache-Control: max-age=600
< Expires: Tue, 01 Jan 2019 02:56:29 GMT
< Last-Modified: Thu, 22 Jun 2017 18:27:09 GMT
< Content-Type: image/jpeg
< Content-Length: 201557
< X-NWS-LOG-UUID: 4243021142118093756 4c4df2ad367f7a7282b8c634e5bb5346
< Access-Control-Allow-Origin: *
< X-Daa-Tunnel: hop_count=3
< X-Cache-Lookup: Hit From Upstream 125.39.46.83, cost 33 ms
< X-Upstream-Info: l2.cloudhy.tcdn.qq.com
< X-Cache-Lookup: Hit From Disktank3
< Accept-Ranges: bytes
< X-Cache-Lookup: Hit From Inner Cluster

13. 指定访问的来源客户端ua

-A 'ua'

代码语言:javascript
复制
[root@sheaBJ ~]# curl -Iv "http://www.xxx.com/ckkhj/index.shtml" -x 1.xxx:80 -A 'Smartphone'
* About to connect() to proxy 1.180.204.168 port 80 (#0)
*   Trying 1.180.204.168...
* Connected to 1.180.204.168 (1.180.204.168) port 80 (#0)
> HEAD http://www.dongao.com/ckkhj/index.shtml HTTP/1.1
> User-Agent: Smartphone
> Host: www.dongao.com
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 301 Moved Permanently
HTTP/1.1 301 Moved Permanently
< Server: NWS_TCloud_S1
Server: NWS_TCloud_S1
< Connection: keep-alive
Connection: keep-alive
< Date: Fri, 08 Mar 2019 03:51:38 GMT
Date: Fri, 08 Mar 2019 03:51:38 GMT
< Content-Length: 39
Content-Length: 39
< Location: http://m.dongao.com/ckkhj/index.shtml
Location: http://m.dongao.com/ckkhj/index.shtml

< 
* Excess found in a non pipelined read: excess = 39 url = /ckkhj/index.shtml (zero-length body)
* Connection #0 to host 1.180.204.168 left intact

14. curl常用头部集合

代码语言:javascript
复制
 -H 'Accept-Encoding:gzip'   测试压缩
 -L                 实现301或者302的跳转
 -H 'Pragma: X-Get-Last-Update-Info'     缓存文件建立时间
 -H 'x_upstream_info=on'			cdn节点回源的链路
 -H 'host:7u2q8y.com3.z0.glb.qiniucdn.com'   绑定hosts
 -H 'range:bytes=0-1000'  或 -r 0-0   测试分片支持
 -H "referer:http://*.yqh5.cn"     验证reffer头部
 -d "x=123"  	模拟post请求
 -H "Origin:http://sheazhang.winyp.cn/"  验证跨域的来源情况,其中O大写
 -w "@curl-time"				各个阶段的耗时时间

特殊文件:

代码语言:javascript
复制
[root@VM_49_231_centos ~]# cat curl-time 
    Time-DNS-lookup:  %{time_namelookup}s\n
   Time-TCP-connect:  %{time_connect}s\n
   Time-SSL-connect:  %{time_appconnect}s\n
      Time-redirect:  %{time_redirect}s\n
  Time-Pre-transfer:  %{time_pretransfer}s\n
Time-First-transfer:  %{time_starttransfer}s\n
                    ----------\n
         Time-total:  %{time_total}s\n

15. httpstat的脚本-time

参考git链接

16. 测试源站是否吐出304的情况

-H 'If-Modified-Since:xxx'

代码语言:javascript
复制
[root@VM_14_42_centos ~]# curl -o 1 -v "http://j.xxx.cc/daobo/js/fabric.js" -x 139.xxx.204:80 -H 'If-Modified-Since: Mon, 23 Apr 2018 03:38:55 GMT'
* About to connect() to proxy 139.199.214.204 port 80 (#0)
*   Trying 139.xxx...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
				 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 139.199.214.204 (139.199.214.204) port 80 (#0)
> GET http://j.xxx.cc/daobo/js/fabric.js HTTP/1.1
> User-Agent: curl/7.29.0
> Host: xxx
> Accept: */*
> Proxy-Connection: Keep-Alive
> If-Modified-Since: Mon, 23 Apr 2018 03:38:55 GMT
>
< HTTP/1.1 304 Not Modified
< Server: nginx/1.14.2
< Date: Mon, 15 Apr 2019 07:40:50 GMT
< Connection: keep-alive
< Last-Modified: Mon, 23 Apr 2018 03:38:55 GMT
< ETag: "5add554f-da835"
< Access-Control-Allow-Origin: *
<
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Connection #0 to host 139.199.214.204 left intact

17. 其他的不常见用法,仅做了解

1、抓取页面内容到一个文件中

代码语言:javascript
复制
[root@www.itts-union.com]# curl -o home.html  http://www.itts-union.com

2、模拟表单信息,模拟登录,保存cookie信息

代码语言:javascript
复制
[root@www.itts-union.com]# curl -c ./cookie_c.txt -F log=aaaa -F pwd=****** http://www.itts-union.com/wp-login.php

3、使用cookie文件

代码语言:javascript
复制
[root@www.itts-union.com]# curl -b ./cookie_c.txt  http://www.itts-union.com/wp-admin

使用cookie头部:

代码语言:javascript
复制
[root@www.itts-union.com]# curl -o /dev/null -v http://www.itts-union.com/wp-admin -H "Cookie: uid=999;sex=2"

4、显示下载进度条

代码语言:javascript
复制
[root@VM_117_214_centos ~]# curl -o smokeping.tar.gz http://oss.oetiker.ch/smokeping/pub/smokeping-2.6.9.tar.gz -#
######################################################################## 100.0%

5、通过ftp下载文件

代码语言:javascript
复制
[zhangy@www.itts-union.com ~]$ curl -u 用户名:密码 -O http://www.itts-union.com/demo/curtain/bbstudy_files/style.css
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
  Dload  Upload   Total   Spent    Left  Speed
  101  1934  101  1934    0     0   3184      0 –:–:– –:–:– –:–:–  7136

   或者用下面的方式

代码语言:javascript
复制
[zhangy@www.itts-union.com ~]$ curl -O ftp://用户名:密码@ip:port/demo/curtain/bbstudy_files/style.css
[zhangy@www.itts-union.com ~]$ curl -O ftp://用户名:密码@ip:port/demo/curtain/bbstudy_files/style.css

6、通过ftp上传

代码语言:javascript
复制
[zhangy@www.itts-union.com ~]$ curl -T test.sql ftp://用户名:密码@ip:port/demo/curtain/bbstudy_files/

7、如下载网页发现乱码,可以使用iconv转码:

代码语言:javascript
复制
[root@VM_117_214_centos
 ~]# curl -o smokeping.tar.gz 
http://oss.oetiker.ch/smokeping/pub/smokeping-2.6.9.tar.gz -# | iconv 
-fgb2312
######################################################################## 100.0%

8、验证https访问

代码语言:javascript
复制
curl -sv 'https://mccdn.qcloud.com/static/img/57c45bb3997a277e5996dcf13314b365/user-40.png' -o/dev/null --resolve "mccdn.qcloud.com:443:221.204.60.11"

get back

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 前言
    • 1. 验证某节点ip缓存情况
      • 2. 验证referer头部白名单是否生效
        • 3. 验证节点/源站对分片的支持情况
          • 4. 验证文件在节点和源站上大小是否一致(通过md5判断)
            • 5. 模拟https访问验证https证书
              • 6. curl中绑定host测试方法
                • 7. 验证TCP握手和ssl握手的延时
                  • 8. 使用curl抓取返回时间
                    • 9. 查询各个阶段的耗时时间:
                      • 10. -L实现301或者302的跳转
                        • 11. 缓存文件建立时间
                          • 12. 查看cdn节点回源的链路
                            • 13. 指定访问的来源客户端ua
                              • 14. curl常用头部集合
                                • 15. httpstat的脚本-time
                                  • 16. 测试源站是否吐出304的情况
                                    • 17. 其他的不常见用法,仅做了解
                                    相关产品与服务
                                    命令行工具
                                    腾讯云命令行工具 TCCLI 是管理腾讯云资源的统一工具。使用腾讯云命令行工具,您可以快速调用腾讯云 API 来管理您的腾讯云资源。此外,您还可以基于腾讯云的命令行工具来做自动化和脚本处理,以更多样的方式进行组合和重用。
                                    领券
                                    问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档