ORA-00254/ORA-15173 Archive_log Directory On Asm Being Deleted

1、错误信息
Thu Nov 29 10:11:22 2012
Errors in file /u01/oracle/admin/SY5221/bdump/sy5221a_arc0_12650.trc:
ORA-16038: log 3 sequence# 2876 cannot be archived
ORA-00254: error in archive control string ''
ORA-00312: online log 3 thread 1: '+DG1/sy5221/onlinelog/log3asy5221.log'
ORA-00312: online log 3 thread 1: '+DG1/sy5221/onlinelog/log3bsy5221.log'
ORA-15173: entry 'archivelog' does not exist in directory 'sy5221'

2、演示环境
SQL> select * from v$version where rownum<2;

BANNER
----------------------------------------------------------------
Oracle Database 10g Release 10.2.0.3.0 - 64bit Production

SQL> select inst_id,instance_name from gv$instance; -->两节点RAC

   INST_ID INSTANCE_NAME
---------- ----------------
         1 GOBO4A
         2 GOBO4B

SQL> show parameter log_archive_dest_1

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_1                   string      LOCATION=+REV/GOBO4/archivelog

3、模拟类似情形         
ASMCMD> pwd
+REV/GOBO4/ARCHIVELOG
ASMCMD> ls       --#当前路径下存在一个文件夹和一个归档日志
2012_11_30/
arch_795194241_1_296.arc
ASMCMD> rm -rf 2012_11_30/  --#删除归档日志文件夹，此时对应的归档日志arch_795194241_1_296.arc也被删除
ASMCMD> ls                  --#再次查看时尽然提示目录GOBO4不存在，汗
asmcmd: entry 'GOBO4' does not exist in directory '+REV/'  

SQL> alter system archive log current;   --#尝试切换日志，已经无法归档
alter system archive log current
*
ERROR at line 1:
ORA-16014: log 6 sequence# 151 not archived, no available destinations
ORA-00312: online log 6 thread 2: '+DG1/gobo4/onlinelog/group_6.256.795194043'

oracle@bo2dbp:/u01/database/GOBO4/bdump> tail -20 alert_GOBO4A.log   --#查看告警日志，
Fri Nov 30 16:08:22 2012                                      --#ORA-15173: entry 'gobo4' does not exist in directory '/'
Errors in file /u01/database/GOBO4/bdump/gobo4a_arc0_5419.trc:
ORA-16038: log 1 sequence# 297 cannot be archived
ORA-00254: error in archive control string ''
ORA-00312: online log 1 thread 1: '+DG1/gobo4/onlinelog/log1agobo4.log'
ORA-00312: online log 1 thread 1: '+DG1/gobo4/onlinelog/log1bgobo4.log'
ORA-15173: entry 'gobo4' does not exist in directory '/'
Fri Nov 30 16:08:25 2012
ARCH: Archival stopped, error occurred. Will continue retrying
Fri Nov 30 16:08:25 2012
ORACLE Instance GOBO4A - Archival Error
Fri Nov 30 16:08:25 2012
ORA-16014: log 1 sequence# 297 not archived, no available destinations
ORA-00312: online log 1 thread 1: '+DG1/gobo4/onlinelog/log1agobo4.log'
ORA-00312: online log 1 thread 1: '+DG1/gobo4/onlinelog/log1bgobo4.log'
Fri Nov 30 16:08:25 2012
Errors in file /u01/database/GOBO4/bdump/gobo4a_arc1_5421.trc:
ORA-16014: log 1 sequence# 297 not archived, no available destinations
ORA-00312: online log 1 thread 1: '+DG1/gobo4/onlinelog/log1agobo4.log'
ORA-00312: online log 1 thread 1: '+DG1/gobo4/onlinelog/log1bgobo4.log'

回到原来的asmcmd命令符提示窗口
ASMCMD> pwd
+REV/GOBO4/ARCHIVELOG
ASMCMD> exit      --#退出asmcmd 
oracle@bo2dbp:~> export ORACLE_SID=+ASM1
oracle@bo2dbp:~> asmcmd      --#再次登陆到asmcmd，果然GOBO4以其下的archivelog文件都不存在了，这下子玩大了，哈哈
ASMCMD> cd +REV/GOBO4        --#仅仅删除归档日志，结果连同上级目录都给干翻了。
asmcmd: entry 'GOBO4' does not exist in directory '+REV/'

4、解决问题
既然目录丢失，那就先创建再次尝试归档
ASMCMD> cd REV
ASMCMD> ls
GOBO1/
ORA10G/
ASMCMD> mkdir GOBO4
ASMCMD> cd GOBO4
ASMCMD> mkdir archivelog
ASMCMD> pwd
+REV/GOBO4/archivelog
ASMCMD> ls

回到sqlplus窗口，依然提示归档失败
SQL> alter system archive log current;
alter system archive log current
*
ERROR at line 1:
ORA-16014: log 6 sequence# 151 not archived, no available destinations
ORA-00312: online log 6 thread 2: '+DG1/gobo4/onlinelog/group_6.256.795194043'

SQL> startup force;   --#直接重启当前节点
ORACLE instance started.

Total System Global Area  536870912 bytes
Fixed Size                  2074080 bytes
Variable Size             432015904 bytes
Database Buffers           96468992 bytes
Redo Buffers                6311936 bytes
Database mounted.
Database opened.
SQL> alter system archive log current;   --#再次归档成功

System altered.

#Author : Robinson
#Blog : http://blog.csdn.net/robinson_0612

ASMCMD> ls
2012_11_30/
arch_795194241_1_297.arc
arch_795194241_1_298.arc
arch_795194241_1_299.arc
arch_795194241_1_300.arc
arch_795194241_2_151.arc
arch_795194241_2_152.arc
arch_795194241_2_153.arc

5、Oracle Bug
metalink上，ID 343784.1给出了关于该问题的描述，同时也给出了该错误的地bug号与解决方法

Archive_log Directory On Asm Being Deleted By Rman Archivelog Backup [ID 343784.1]
Bug 4431215 "BACKUP ARCHIVELOG ALL DELETE INPUT" FROM RMAN REMOVES ASM DIRECTORY

To implement the solution, please execute either of the following steps:

1. Change the log_archive_dest to NOT include the SID in the ASM directory name.
OR
2. Create a subdirectory under the ARCHIVELOG directory. As RMAN will not delete a directory which contains files 
  (or directories), having an empty subdirectory will prevent RMAN from deleting.  

也就是说，如果archivelog下一旦为空的情形下，不论是手动删除，还是使用RMAN，Oracle会自动删除该目录。
文中开头部分描述的错误信息entry 'archivelog' does not exist in directory 'sy5221'即是如此。
后来的演示环境中，除了删除archivelog外，连同上级目录GOBO4目录也删除了，因为GOBO4下没有任何文件或非空目录
针对这种情形，Oracle建议修改归档路径，且路径中的文件夹不能包含ORACLE_SID，其次也可以通过在归档路径最底层创建一个非空目录。

我的解决方式是备份一个spfile到archivelog下,哈哈。
SQL> create pfile='/u01/oracle/db/dbs/initGOBO4A.ora.bak' from spfile;

File created.

SQL> create spfile='+REV/GOBO4/archivelog/spfileGOBO4.bak' from
  2  pfile='/u01/oracle/db/dbs/initGOBO4A.ora.bak';

File created.

RMAN下，试试delete noprompt archivelog all，(不要在生产环境这么干，出了事偶不负责任阿，嘎嘎。。)