Oracle数据库恢复:归档日志损坏案例一则

链接:http://www.eygle.com/archives/2010/11/recover_archivelog_corruption.html

最近在紧急故障处理时,帮助用户恢复数据库遇到了一则罕见的归档日志损坏案例,在这里和大家分享一下,看看是否有人遇到过类似的问题。 在进行归档recover时,数据库报错,提示归档日志损坏:

*** Corrupt block seq: 37288 blocknum=1. Bad header found during deleting archived log Data in bad block - seq:810559520. bno:170473264. time:707406346 beg:21280 cks:21061 calculated check value: 9226 Reread of seq=37288, blocknum=1, file=/ARCH/arch_1_37288_632509987.dbf, found same corrupt data Reread of seq=37288, blocknum=1, file=/ARCH/arch_1_37288_632509987.dbf, found same corrupt data Reread of seq=37288, blocknum=1, file=/ARCH/arch_1_37288_632509987.dbf, found same corrupt data Reread of seq=37288, blocknum=1, file=/ARCH/arch_1_37288_632509987.dbf, found same corrupt data Reread of seq=37288, blocknum=1, file=/ARCH/arch_1_37288_632509987.dbf, found same corrupt data ***

信息比较详细,说37288号归档日志Header损坏,无法读取数据。 提一个小问题:如果你遇到了这样的错误?会怎样思考? 如果这个归档日志损坏了,其实我们仍然有办法跳过去,继续尝试恢复其他日志,但是客户数据重要,不能容忍不一致性,这时候就只能放弃部分数据,由前台重新提交数据了。这在业务上可以实现,也就不是大问题了。 好了,问题是为什么日志会损坏?是如何损坏的? 我首先要做的就是,看看日志文件的内容,通过最简单的命令将日志文件中的内容输出出来: strings arch_1_37288_632509987.dbf > log.txt 然后检查生成的这个日志文件,我们就发现了问题。 在这个归档日志文件中,被写入了大量的跟踪文件内容,其中开头部分就是一个跟踪文件的全部信息。 这是一种我从来没有遇到过的现象,也就是说,当操作系统在写出跟踪文件时,错误的覆盖掉了已经存在的归档文件,最后导致归档日志损坏,非常奇妙,从所未见。 最后我的判断是,这个故障应当是操作系统在写出时出现了问题,存在文件的空间仍然被认为是可写的,这样就导致了写冲突,出现这类问题,应当立即检查硬件,看看是否是硬件问题导致了如此严重的异常。

Dump file /ADMIN/bdump/erp_p007_19216.trc Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - Production With the Partitioning, OLAP and Data Mining options ORACLE_HOME = /DBMS/erp/erpdb/10g Linux eygle.com 2.6.9-34.ELhugemem #1 SMP Fri Feb 24 17:04:34 EST 2006 i686 Instance name: erp Redo thread mounted by this instance: 1 Oracle process number: 22 Unix process pid: 19216, image: oracle@eygle.com (P007) *** SERVICE NAME:() 2010-11-10 10:37:26.247 *** SESSION ID:(2184.1) 2010-11-10 10:37:26.247 *** 2010-11-10 10:37:26.247 KCRP: blocks claimed = 61, eliminated = 0 ----- Recovery Hash Table Statistics --------- Hash table buckets = 32768 Longest hash chain = 1 Average hash chain = 61/61 = 1.0 Max compares per lookup = 0 Avg compares per lookup = 0/61 = 0.0 ---------------------------------------------- ----- Recovery Hash Table Statistics --------- Hash table buckets = 32768 Longest hash chain = 1 Average hash chain = 61/61 = 1.0 Max compares per lookup = 1 Avg compares per lookup = 1426/1426 = 1.0 ---------------------------------------------- /GPAYMENTdxn AP_CHECKS Q(xn .1=N /Gxn .1=N ^0e ^0e! ^0e" ^0e# ^0e$ ^0e% ^0e& ^0e' eygle.com!/ ^0e( ^0e) ^0e* ^0e+ ^0e+ ^0e& ^ij1 R0:b Q(xn PaymentsN a'VND Userxn AP_INVOICE_PAYMENTS 105273 5406105305-20101020-003 3001CASH CLEARING CREATED Dump file /ADMIN/bdump/erp_p002_19206.trc Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - Production With the Partitioning, OLAP and Data Mining options ORACLE_HOME = /DBMS/erp/erpdb/10g Linux eygle.com 2.6.9-34.ELhugemem #1 SMP Fri Feb 24 17:04:34 EST 2006 i686 Instance name: erp Redo thread mounted by this instance: 1 Oracle process number: 17 Unix process pid: 19206, image: oracle@eygle.com (P002) *** SERVICE NAME:() 2010-11-10 10:37:26.263 *** SESSION ID:(2187.1) 2010-11-10 10:37:26.263 *** 2010-11-10 10:37:26.263 KCRP: blocks claimed = 84, eliminated = 0 ----- Recovery Hash Table Statistics --------- Hash table buckets = 32768 Longest hash chain = 1 Average hash chain = 84/84 = 1.0 Max compares per lookup = 0 Avg compares per lookup = 0/84 = 0.0 ---------------------------------------------- ----- Recovery Hash Table Statistics --------- Hash table buckets = 32768 Longest hash chain = 1 Average hash chain = 84/84 = 1.0 Max compares per lookup = 1 Avg compares per lookup = 880/880 = 1.0 ---------------------------------------------- ^A&A ^1b# ^1b! ^1b" ^0e' ^Mj8 ^;&3 2010PS_Legal Entity ^6&L Eoi_VND Quick Payment: ID=47708 Cn/a UNSENT ^9&1 /HPAYMENT CREATEDNAP_CHECKS ^0e) /Hxn Dump file /ADMIN/bdump/erp_p001_19204.trc Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - Production With the Partitioning, OLAP and Data Mining options ORACLE_HOME = /DBMS/erp/erpdb/10g Linux eygle.com 2.6.9-34.ELhugemem #1 SMP Fri Feb 24 17:04:34 EST 2006 i686 Instance name: erp Redo thread mounted by this instance: 1 Oracle process number: 16 Unix process pid: 19204, image: oracle@eygle.com (P001) *** SERVICE NAME:() 2010-11-10 10:37:26.372 *** SESSION ID:(2189.1) 2010-11-10 10:37:26.372 *** 2010-11-10 10:37:26.372 KCRP: blocks claimed = 132, eliminated = 0 ----- Recovery Hash Table Statistics --------- Hash table buckets = 32768 Longest hash chain = 1 Average hash chain = 132/132 = 1.0 Max compares per lookup = 0 Avg compares per lookup = 0/132 = 0.0 ---------------------------------------------- ----- Recovery Hash Table Statistics --------- Hash table buckets = 32768 Longest hash chain = 1 Average hash chain = 132/132 = 1.0 Max compares per lookup = 1 Avg compares per lookup = 3219/3219 = 1.0 ---------------------------------------------- ^ij! ^ij$ ^ij! @e>df >df^>df Userxn Chen Restaurant 300190143 CASH CLEARING AP_CHECKS CREATED ACCOUNTED ^ij! CHECK en/a Quick Payment: ID=47708 n/a^n/a CHECK Chen Restaurant 210301 5&1` 54^` ^1b$ ^1b& ^1b6 ^1b, ^1b- ^1b4 ^1b5 ^1b0 ^1b2 Dump file /ADMIN/bdump/erp_p000_19202.trc Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - Production With the Partitioning, OLAP and Data Mining options ORACLE_HOME = /DBMS/erp/erpdb/10g Linux eygle.com 2.6.9-34.ELhugemem #1 SMP Fri Feb 24 17:04:34 EST 2006 i686 Instance name: erp Redo thread mounted by this instance: 1 Oracle process number: 15 Unix process pid: 19202, image: oracle@eygle.com (P000) *** SERVICE NAME:() 2010-11-10 10:37:26.386 *** SESSION ID:(2190.1) 2010-11-10 10:37:26.386 *** 2010-11-10 10:37:26.386 KCRP: blocks claimed = 181, eliminated = 0 ----- Recovery Hash Table Statistics --------- Hash table buckets = 32768 Longest hash chain = 1 Average hash chain = 181/181 = 1.0 Max compares per lookup = 0 Avg compares per lookup = 0/181 = 0.0 ---------------------------------------------- ----- Recovery Hash Table Statistics --------- Hash table buckets = 32768 Longest hash chain = 1 Average hash chain = 181/181 = 1.0 Max compares per lookup = 1 Avg compares per lookup = 8629/8629 = 1.0 ---------------------------------------------- ^ij0 AGENT_STATUS_MARKER ^AGENTS_MARKED R0:b ^E! ^1b ^1b ^1b! ^1b! ^1b" ^1b" ^1b#

如此少见的案例,在此与大家分享。

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏bboysoul

linux编译安装apache

wget http://mirrors.ustc.edu.cn/apache/httpd/httpd-2.4.25.tar.gz tar -zxvf http...

35530
来自专栏杨建荣的学习笔记

生产环境sqlldr加载性能问题及分析之二(r2第20天)

上一节讨论了在数据迁移中发现数据加载的速度一下子慢了很多,和之前在测试环境相比有很大的差距。一个原因就是由于在数据加载的过程中有一些额外的session也在操作...

30650
来自专栏大大的微笑

深入理解JVM原理之编译openjdk7

Java虚拟机种类很多例如:HotSpot 丶 JRockit 丶 J9等 我们一般用的就是HotSpot,如果我们电脑上装了JDK,可以运行java -ver...

31490
来自专栏微信公众号:Java团长

详解Intellij IDEA搭建SpringBoot

Spring Boot是由Pivotal团队提供的全新框架,其设计目的是用来简化新Spring应用的初始搭建以及开发过程。该框架使用了特定的方式来进行配置,从而...

1.1K40
来自专栏杨建荣的学习笔记

oracle工具集初探(r4笔记第8天)

今天无意中看了下ORACLE_HOME/bin下面的东西,发现里面还是存在不少的东西。除了常用的sqlplus,tnsping,rman,exp/expdp,i...

32640
来自专栏杨建荣的学习笔记

由一条日志警告所做的调优分析(r3笔记第40天)

这个案例发生有段时间了,但是今天无意中看到当时的邮件,感觉还是收益匪浅,看来还是细节决定成败啊。从一些日志或trace 文件中的警告信息中我们可以发掘出潜在的问...

38130
来自专栏杨建荣的学习笔记

关于CPU使用率高的awr分析(r8笔记第46天)

今天看到一个报警信息,大体是CPU使用异常。 ZABBIX-监控系统: ------------------------------------ 报警内容: ...

34950
来自专栏耕耘实录

找回win7桌面IE图标我有绝招

现在很多同学都用了微软的新一代操作系统Windows7,都为Windows7的华丽界面及更人性化得操作所深深吸引,但是由于我们大多数同学都习惯了原来的Windo...

40550
来自专栏杨建荣的学习笔记

11g Dataguard中的snapshot standby特性(r8笔记第49天)

11g中的ADG特性本身已经非常有特色,促使很多对于10g中不太灵便的备库升级到11g,对于DBA是一大福利,那么还有一个福利就是snapshot standb...

32950
来自专栏蓝天

Redis模块开发示例

实现一个Redis module,支持两个扩展命令: 1) 可同时对hash的多个field进行incr操作; 2) incrby同时设置一个key的过期时...

15130

扫码关注云+社区

领取腾讯云代金券