文章/答案/技术大牛

发布

社区首页 >问答首页 >ParserError error=模式与数据不匹配

问ParserError error=模式与数据不匹配
EN

Stack Overflow用户

提问于 2022-04-01 10:07:59

回答 2查看 1.2K关注 0票数 2

我试图将Kubernetes日志从容器重定向到OpenSearch。但是，日期总是有一些错误。我做错了什么？

Docker日志示例：

{"log":"time=\"2022-04-01T10:02:31Z\" level=warning msg=\"Cannot take snapshot backup\" controller=longhorn-backup error=\"could not find snapshot 'snapshot-0d1744c2-ff8d-4a68-8a2c-fbfd16408975' to backup, volume 'pvc-03557105-c20d-4fbe-8d0d-8a0b4ac16f6d'\" node=k8s-worker2\n","stream":"stderr","time":"2022-04-01T10:02:31.230191143Z"}
{"log":"E0401 10:02:31.230146       1 backup_controller.go:153] longhorn-backup: fail to sync backup longhorn-system/backup-989764daba094e0d: could not find snapshot 'snapshot-0d1744c2-ff8d-4a68-8a2c-fbfd16408975' to backup, volume 'pvc-03557105-c20d-4fbe-8d0d-8a0b4ac16f6d'\n","stream":"stderr","time":"2022-04-01T10:02:31.230214608Z"}
{"log":"time=\"2022-04-01T10:02:31Z\" level=warning msg=\"Dropping Longhorn backup longhorn-system/backup-989764daba094e0d out of the queue\" controller=longhorn-backup error=\"longhorn-backup: fail to sync backup longhorn-system/backup-989764daba094e0d: could not find snapshot 'snapshot-0d1744c2-ff8d-4a68-8a2c-fbfd16408975' to backup, volume 'pvc-03557105-c20d-4fbe-8d0d-8a0b4ac16f6d'\" node=k8s-worker2\n","stream":"stderr","time":"2022-04-01T10:02:31.230218285Z"}

流量产出：

fluentd/fluentd-x7pgc[fluentd]: 2022-04-01 09:54:33 +0000 [warn]: #0 dump an error event: error_class=Fluent::Plugin::Parser::ParserError error="pattern not matched with data 'time=\"2022-04-01T09:54:33Z\" level=warning msg=\"Cannot take snapshot backup\" controller=longhorn-backup error=\"could not find snapshot 'snapshot-15dfec02-b8c4-40db-a7ed-bf84429ac220' to backup, volume 'pvc-03557105-c20d-4fbe-8d0d-8a0b4ac16f6d'\" node=k8s-worker1\n'" location=nil tag="kubernetes.var.log.containers.longhorn-manager-5lmdf_longhorn-system_longhorn-manager-5f6bc9870a9efe75670274d177d0bf17dee0dd995a433343432b3155af946823.log" time=2022-04-01 09:54:33.524389862 +0000 record={"log"=>"time=\"2022-04-01T09:54:33Z\" level=warning msg=\"Cannot take snapshot backup\" controller=longhorn-backup error=\"could not find snapshot 'snapshot-15dfec02-b8c4-40db-a7ed-bf84429ac220' to backup, volume 'pvc-03557105-c20d-4fbe-8d0d-8a0b4ac16f6d'\" node=k8s-worker1\n", "stream"=>"stderr"}
fluentd/fluentd-x7pgc[fluentd]: 2022-04-01 09:54:33 +0000 [warn]: #0 dump an error event: error_class=Fluent::Plugin::Parser::ParserError error="pattern not matched with data 'time=\"2022-04-01T09:54:33Z\" level=warning msg=\"Error syncing Longhorn backup longhorn-system/backup-c5104cf80da04be6\" controller=longhorn-backup error=\"longhorn-backup: fail to sync backup longhorn-system/backup-c5104cf80da04be6: could not find snapshot 'snapshot-15dfec02-b8c4-40db-a7ed-bf84429ac220' to backup, volume 'pvc-03557105-c20d-4fbe-8d0d-8a0b4ac16f6d'\" node=k8s-worker1\n'" location=nil tag="kubernetes.var.log.containers.longhorn-manager-5lmdf_longhorn-system_longhorn-manager-5f6bc9870a9efe75670274d177d0bf17dee0dd995a433343432b3155af946823.log" time=2022-04-01 09:54:33.524404952 +0000 record={"log"=>"time=\"2022-04-01T09:54:33Z\" level=warning msg=\"Error syncing Longhorn backup longhorn-system/backup-c5104cf80da04be6\" controller=longhorn-backup error=\"longhorn-backup: fail to sync backup longhorn-system/backup-c5104cf80da04be6: could not find snapshot 'snapshot-15dfec02-b8c4-40db-a7ed-bf84429ac220' to backup, volume 'pvc-03557105-c20d-4fbe-8d0d-8a0b4ac16f6d'\" node=k8s-worker1\n", "stream"=>"stderr"}
fluentd/fluentd-x7pgc[fluentd]: 2022-04-01 09:54:33 +0000 [warn]: #0 dump an error event: error_class=Fluent::Plugin::Parser::ParserError error="pattern not matched with data 'time=\"2022-04-01T09:54:33Z\" level=warning msg=\"Cannot take snapshot backup\" controller=longhorn-backup error=\"could not find snapshot 'snapshot-9d5705bf-26fc-49e5-a771-cf9352049c04' to backup, volume 'pvc-03557105-c20d-4fbe-8d0d-8a0b4ac16f6d'\" node=k8s-worker1\n'" location=nil tag="kubernetes.var.log.containers.longhorn-manager-5lmdf_longhorn-system_longhorn-manager-5f6bc9870a9efe75670274d177d0bf17dee0dd995a433343432b3155af946823.log" time=2022-04-01 09:54:33.538539106 +0000 record={"log"=>"time=\"2022-04-01T09:54:33Z\" level=warning msg=\"Cannot take snapshot backup\" controller=longhorn-backup error=\"could not find snapshot 'snapshot-9d5705bf-26fc-49e5-a771-cf9352049c04' to backup, volume 'pvc-03557105-c20d-4fbe-8d0d-8a0b4ac16f6d'\" node=k8s-worker1\n", "stream"=>"stderr"}

配置：

<source>
  @type tail
  @id tail_all_container_logs
  @label @FLUENTD.OPENSEARCH
  path /var/log/containers/longhorn*.log
  pos_file /var/log/fluentd-containers.log.pos
  tag kubernetes.*
  exclude_path "#{ENV['FLUENT_ALL_CONTAINERS_TAIL_EXCLUDE_PATHS']}"
  <parse>
      @type json
  </parse>
</source>

<filter kubernetes.**>
    @type parser
    key_name log
    <parse>
        @type json
        time_format %Y-%m-%dT%H:%M:%S.%N%z
        timezone +00:00
    </parse>
</filter>

docker

kubernetes

logging

fluentd

fluent

回答 2

Stack Overflow用户

发布于 2022-04-04 17:08:14

日志中的日期/时间是2022-04-01T09:54:33Z，注意不要毫秒。

在您的配置中有time_format %Y-%m-%dT%H:%M:%S.%N%z

%N -分数秒数，默认为9位数(纳秒).

尝试从日志中删除与时间格式匹配的.%N部件。这将是：

time_format %Y-%m-%dT%H:%M:%S%z

有关时间格式语法的更多信息，请参阅此页面。

票数 0

Stack Overflow用户

发布于 2022-04-05 08:55:22

@piotr-malec -谢谢你的回答。我尝试过许多不同的日期/时间格式，但总是一样的。

<filter kubernetes.**>
    @type parser
    key_name log
    <parse>
        @type json
        time_format %Y-%m-%dT%H:%M:%S.%N%z
        #time_format %Y-%m-%dT%H:%M:%S.%z
        #time_format %Y-%m-%dT%H:%M:%S%z
        #time_format %Y-%m-%dT%H:%M:%Sz
        #time_format %Y-%m-%dT%H:%M:%SZ
        #time_format %Y-%m-%dT%H:%M:%S%Z
        #time_format %Y-%m-%dT%H:%M:%S.%z
        timezone +00:00
    </parse>
</filter>

现在，我有一个类似的案例，有一个不同的日志。

fluentd/fluentd-9gfjs[fluentd]: 2022-04-05 08:46:58 +0000 [warn]: #0 dump an error event: error_class=Fluent::Plugin::Parser::ParserError error="pattern not matched with data '[2022-04-05T08:46:58,305][INFO ][o.o.j.s.JobSweeper       ] [opensearch-cluster-master-0] Running full sweep\n'" location=nil tag="opensearch-master" time=1970-01-01 00:33:42.306044198 +0000 record={"log"=>"[2022-04-05T08:46:58,305][INFO ][o.o.j.s.JobSweeper       ] [opensearch-cluster-master-0] Running full sweep\n", "stream"=>"stdout"}
fluentd/fluentd-9gfjs[fluentd]: 2022-04-05 08:51:58 +0000 [warn]: #0 dump an error event: error_class=Fluent::Plugin::Parser::ParserError error="pattern not matched with data '[2022-04-05T08:51:58,306][INFO ][o.o.j.s.JobSweeper       ] [opensearch-cluster-master-0] Running full sweep\n'" location=nil tag="opensearch-master" time=1970-01-01 00:33:42.306370871 +0000 record={"log"=>"[2022-04-05T08:51:58,306][INFO ][o.o.j.s.JobSweeper       ] [opensearch-cluster-master-0] Running full sweep\n", "stream"=>"stdout"}

我尝试过以下格式。他们每一个都不起作用。

<filter opensearch-master>
    @type parser
    key_name log
    <parse>
        @type json
        time_key @timestamp
        time_format %Y-%m-%dT%H:%M:%S,%3N
        #time_format %Y-%m-%dT%H:%M:%S,%L
        timezone +02:00
    </parse>
</filter>

Ruby文档：https://docs.ruby-lang.org/en/2.4.0/Time.html#method-i-strftime

票数 0

页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持

原文链接：

https://stackoverflow.com/questions/71705233

复制

相似问题

问ParserError error=模式与数据不匹配
EN

回答 2

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问ParserError error=模式与数据不匹配EN

回答 2

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问ParserError error=模式与数据不匹配
EN