ORA-27090 故障一例

    最近的alert日志中碰到了ORA-27090的错误信息，其错误提示为Unable to reserve kernel resources for asynchronous disk I/O。根据这个提示来看是跟异步I/O相关的内核参数问题。下面是这个问题的描述与解决。

1、故障错误信息 Wed Sep 10 09:58:17 CST 2014 Errors in file /u01/app/oracle/admin/orclls/bdump/orclls_p000_47020.trc: ORA-27090: Unable to reserve kernel resources for asynchronous disk I/O Linux-x86_64 Error: 2: No such file or directory Additional information: 3 Additional information: 128 Additional information: 65536 Wed Sep 10 09:58:17 CST 2014 Errors in file /u01/app/oracle/admin/orclls/bdump/orclls_p002_47024.trc: ORA-27090: Unable to reserve kernel resources for asynchronous disk I/O Linux-x86_64 Error: 2: No such file or directory Additional information: 3 Additional information: 128 Additional information: 65536  

2、故障分析 #调用oerr，没有获得比较有价值的信息，如下： suse11b:oracle:orcl102 > oerr ora 27090 27090, 00000, "Unable to reserve kernel resources for asynchronous disk I/O" // *Cause:  The system call to reserve kernel resources for asynchronous I/O //          has failed. // *Action: Check errno

#进一步查看trace文件，注，当前数据库版本为10.2.0.5，OS为suse 11 sp3 suse11a:oracle:orcl101 > more /u01/app/oracle/admin/orclls/bdump/orclls_p002_47024.trc /u01/app/oracle/admin/orclls/bdump/orclls_p002_47024.trc Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production With the Partitioning, Real Application Clusters, OLAP, Data Mining and Real Application Testing options ORACLE_HOME = /u01/app/oracle/product/10.2/db_1 System name:    Linux Node name:      suse11a Release:        3.0.76-0.11-default Version:        #1 SMP Fri Jun 14 08:21:43 UTC 2013 (ccab990) Machine:        x86_64 Instance name: orclls Redo thread mounted by this instance: 1 Oracle process number: 35 Unix process pid: 47024, image: oracle@suse11a (P002)

*** SERVICE NAME:() 2014-09-10 09:58:17.159 *** SESSION ID:(1072.1) 2014-09-10 09:58:17.159 *** 2014-09-10 09:58:17.159 KCRP: blocks claimed = 0, eliminated = 0 ----- Recovery Hash Table Statistics --------- Hash table buckets = 32768 Longest hash chain = 0 Average hash chain = 0/0 = 0.0 Max compares per lookup = 0 Avg compares per lookup = 0/0 = 0.0 ---------------------------------------------- ORA-27090: Unable to reserve kernel resources for asynchronous disk I/O Linux-x86_64 Error: 2: No such file or directory Additional information: 3 Additional information: 128 Additional information: 65536 ----- Recovery Hash Table Statistics --------- Hash table buckets = 32768 Longest hash chain = 0 Average hash chain = 0/0 = 0.0 Max compares per lookup = 0 Avg compares per lookup = 0/0 = 0.0 ---------------------------------------------- #从trace文件中也没有获得比较有价值的信息

3、故障解决 #根据ORA-27090: MESSAGE 27090 NOT FOUND; (文档 ID 579108.1)文档，我们修改内核参数fs.aio-max-nr #由于此数据库为RAC环境下，因此同时调整了2个节点，以下为调整后的情形。 suse11a:~ # grep max-nr /etc/sysctl.conf #fs.aio-max-nr = 1048576 fs.aio-max-nr = 3145728

suse11b:~ # grep max-nr /etc/sysctl.conf #aio-max-nr = 1048576 fs.aio-max-nr = 3145728

suse11a:~ # sysctl -p suse11b:~ # sysctl -p

#配置完毕后可以通过cluvy校验 #几日后观察alert日志，未发现该类错误。

Check: Kernel parameter for "aio-max-nr"   Node Name         Current       Configured    Required      Status        Comment       ----------------  ------------  ------------  ------------  ------------  ------------   suse11a           3145728       3145728       1048576       passed            suse11b           3145728       3145728       1048576       passed          Result: Kernel parameter check passed for "aio-max-nr"