前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >clickhouse导入原始nginx日志数据

clickhouse导入原始nginx日志数据

作者头像
仙士可
发布2023-02-23 08:54:29
5830
发布2023-02-23 08:54:29
举报
文章被收录于专栏:仙士可博客仙士可博客

原始日志数据

代码语言:javascript
复制
139.224.56.94 - - [03/Dec/2022:16:01:01 +0800] "GET /output/zw/updateConsumeMoney?ad_num=1740763056491527&date=2022-12-03&cost=0&balance=0&sign=09a77e8f235f45b087a2d15a1b77f430 HTTP/1.0" 200 20 "-" "-"
49.93.209.3 - - [03/Dec/2022:16:01:01 +0800] "GET /index/Quickapi1/read?ref=read&id=1029393&bookid=3109 HTTP/2.0" 200 14547 "https://quickapp.cn/com.hmydb.zzxd/29/page-frame.html" "Mozilla/5.0 (Linux; Android 12; ELS-AN00 Build/HUAWEIELS-AN00;)AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/18.0.1025 Mobile Safari/537.36 hap/1102/huawei com.huawei.fastapp/12.6.1.304 com.hmydb.zzxd/2.0.9 ({\x22packageName\x22:\x22unknown\x22,\x22type\x22:\x22unknown\x22,\x22extra\x22:\x22{}\x22})"
139.224.56.94 - - [03/Dec/2022:16:01:02 +0800] "GET /output/zw/updateConsumeMoney?ad_num=1740763057082376&date=2022-12-03&cost=0&balance=0&sign=321a5fc2b7470896ca9e0991b3dcff7c HTTP/1.0" 200 0 "-" "-"
171.214.147.1 - - [03/Dec/2022:16:01:02 +0800] "GET /index/Quickapi1/getGender HTTP/2.0" 200 26 "https://quickapp.cn/com.hmydb.zzxd/29/page-frame.html" "Mozilla/5.0 (Linux; Android 12; NOH-AN00 Build/HUAWEINOH-AN00;)AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/18.0.1025 Mobile Safari/537.36 hap/1102/huawei com.huawei.fastapp/12.6.1.304 com.hmydb.zzxd/2.0.9 ({\x22type\x22:\x22shortcut\x22,\x22packageName\x22:\x22com.huawei.android.launcher\x22,\x22extra\x22:{\x22isNative\x22:false,\x22scene\x22:\x22api\x22}})"
139.224.56.94 - - [03/Dec/2022:16:01:02 +0800] "GET /output/zw/updateConsumeMoney?ad_num=1740763057797133&date=2022-12-03&cost=0&balance=0&sign=fd36a029809d9d4ddf45da2e9544ad69 HTTP/1.0" 200 0 "-" "-"

创建表:

代码语言:javascript
复制
 CREATE TABLE nginx_log (
               remote_addr String,
               time_local String,
               request String,
               status String,
               body_bytes_sent String,
               http_referer String,
               http_user_agent String
           ) ENGINE = Log;

导入命令:

代码语言:javascript
复制
INSERT INTO nginx_log FROM INFILE 'nginx.log'
           SETTINGS
             format_regexp = '([0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+) - - \\[([0-9]+\\/[a-z,A-Z]+\\/[0-9]+:[0-9]+:[0-9]+:[0-9]+ \\+[0-9]+)\\] "(.+?)" ([0-9]+) ([0-9]+) "(.+?)" "(.+?)"' , format_regexp_skip_unmatched = 1
           FORMAT Regexp
本文参与 腾讯云自媒体同步曝光计划,分享自作者个人站点/博客。
原始发表:2023-02-22,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体同步曝光计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 原始日志数据
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档