ORA-00245: control file backup failed; in Oracle RAC, target might not be on shared storage

RAC database backup with RMAN failed with error:

Starting Control File and SPFILE Autobackup at 2015-09-14:23:13:44
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of Control File and SPFILE Autobackup command on Disk channel at 2015-09-14:23:13:45
ORA-00245: control file backup failed; in Oracle RAC, target might not be on shared storage

This is because From 11gR2 onwards, the controlfile backup happens without holding the controlfile enqueue. For non-RAC database, this doesn't change anything. But for RAC database, due to the changes made to the controlfile backup mechanism in 11gR2, any instance in the cluster may write to the snapshot controlfile. Due to this snapshot controlfile need to be visible to all instances. The snapshot controlfile MUST be accessible by all nodes of a RAC database, if the snapshot controlfile does not reside on a shared device error will be raised at the time of RMAN backup while taking snapshot of controlfile. This applies to backing up controlfile using sqlplus / having autobackup of controlfile configured on non shared location.
Solution:
This is a RAC specific configuration issue and the correct configuration is as described below It is changed behaviour which requires that the snapshot controlfile in a RAC environment, is on a shared location.
Check the snapshot controlfile location:

RMAN> show snapshot controlfile name;

Configure the snapshot controlfile to a shared disk :

RMAN> CONFIGURE SNAPSHOT CONTROLFILE NAME TO '+DG01/snapcf_RACDB.f';

RMAN warnings RMAN-06207, RMAN-06208 due to mismatched status for snapshot controlfile

Hello, Today I faced next warnings in database backup log :

RMAN-06207: WARNING: 1 objects could not be deleted for DISK channel(s) due
RMAN-06208:          to mismatched status.  Use CROSSCHECK command to fix status
RMAN-06210: List of Mismatched objects
RMAN-06211: ==========================
RMAN-06212:   Object Type   Filename/Handle
RMAN-06213: --------------- ---------------------------------------------------
RMAN-06214: Datafile Copy   /u01/app/oracle/product/11.2.0/dbhome_1/dbs/snapc_testdb.f

To solve this warnings we need to change snapshot controlfile name to another file and after that delete mismatched file.

rman target /

Recovery Manager: Release 11.2.0.3.0 - Production on Wed Jul 19 18:14:56 2017

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

connected to target database: TESTDB (DBID=2665187923)

RMAN> show all;

using target database control file instead of recovery catalog
RMAN configuration parameters for database with db_unique_name TESTDB are:
CONFIGURE RETENTION POLICY TO REDUNDANCY 1; # default
CONFIGURE BACKUP OPTIMIZATION OFF; # default
CONFIGURE DEFAULT DEVICE TYPE TO DISK;
CONFIGURE CONTROLFILE AUTOBACKUP ON;
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE 'DISK' TO '%F';
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F'; # default
CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE MAXSETSIZE TO UNLIMITED; # default
CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
CONFIGURE COMPRESSION ALGORITHM 'BASIC' AS OF RELEASE 'DEFAULT' OPTIMIZE FOR LOAD TRUE ; # default
CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/u01/app/oracle/product/11.2.0/dbhome_1/dbs/snapc_testdb.f';

RMAN> CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/u01/app/oracle/product/11.2.0/dbhome_1/dbs/snapc_testdb1.f';
RMAN> crosscheck copy;
RMAN> delete expired copy;

ORA-00230: operation disallowed: snapshot control file enqueue unavailable

Backup job failed because RMAN cannot make a snapshot control file. The message stack is as follows:

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup command at 06/25/2017 22:48:44
ORA-00230: operation disallowed: snapshot control file enqueue unavailable

When RMAN needs to back up or resynchronize from the control file, it first creates a snapshot or consistent image of the control file. If one RMAN job is already backing up the control file while another needs to create a new snapshot control file, then you may see the following message:

waiting for snapshot control file enqueue

To determine which job is holding the conflicting enqueue:

SELECT s.SID,
       USERNAME   AS "User",
       PROGRAM,
       MODULE,
       ACTION,
       LOGON_TIME "Logon",
       l.*,
       'alter system kill session '||''''||s.SID||','||s.SERIAL#||','||'@'||s.INST_ID||''''||' immediate;'
  FROM GV$SESSION s, GV$ENQUEUE_LOCK l
 WHERE l.SID = s.SID
   AND l.TYPE = 'CF'
   AND l.ID1 = 0
   AND l.ID2 = 2;

You should see output similar to the following:

SID User Program              Module                    Action           Logon
--- ---- -------------------- ------------------- ---------------- ---------
18 SYS rman@testdb (TNS V1-V3) backup full datafile: c30000110 STARTED 25-JUN-17

After you have determined which job is creating the enqueue, you can do one of the following:

• Wait until the job holding the enqueue completes
• kill current job and execute backup again

ORA-12720: operation requires database is in EXCLUSIVE mode

When I try to create a controlfile for a RAC database using the CONTROLFILE Trace. It threw the below error message:

CREATE CONTROLFILE .................................
   .................................................
CHARACTER SET AL32UTF8;

ERROR at line 1:
ORA-01503: CREATE CONTROLFILE failed
ORA-12720: operation requires database is in EXCLUSIVE mode

Solution:

SQL> alter system set cluster_database=FALSE scope=spfile sid='*';
shutdown immdeiate;
startup nomount;

re-run script for controlfile creation

CREATE CONTROLFILE .................................
   .................................................
CHARACTER SET AL32UTF8;

Control file created.

after that change parameter with its old value

SQL> alter system set cluster_database=TRUE scope=spfile sid='*';

and restart database.

ORA-19606: Cannot copy or restore to snapshot control file

Hello,
when delete obsolete backups and archivelogs appears next error:

RMAN> delete obsolete;
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of delete command on ORA_DISK_1 channel at 12/05/2013 15:27:49
ORA-19606: Cannot copy or restore to snapshot control file

First of all check rman permanent parameters:

RMAN> show all;
RMAN configuration parameters for database with db_unique_name ORCL are:
CONFIGURE RETENTION POLICY TO REDUNDANCY 1; # default
CONFIGURE BACKUP OPTIMIZATION OFF; # default
CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default
CONFIGURE CONTROLFILE AUTOBACKUP OFF; # default
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F'; # default
CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE MAXSETSIZE TO UNLIMITED; # default
CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
CONFIGURE COMPRESSION ALGORITHM 'BASIC' AS OF RELEASE 
'DEFAULT' OPTIMIZE FOR LOAD TRUE ; # default
CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
CONFIGURE SNAPSHOT CONTROLFILE NAME TO 
'/u01/app/oracle/product/11.2.0/dbhome_1/dbs/snapcf_orcl1.f'; # default

Cause: A control file copy or restore operation specified the name of the
snapshot control file as the output file. It is not permitted to
overwrite the snapshot control file in this manner. Other
methods are available to create the snapshot control file.
Action: Specify a different file name and retry the operation. If this
is a restore, then the restore conversation remains active and
more files may be specified.
Solution:

RMAN> CONFIGURE SNAPSHOT CONTROLFILE NAME TO 
'/u01/app/oracle/product/11.2.0/dbhome_1/dbs/snap_orcl1.f';
new RMAN configuration parameters:
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/u01/app/oracle/product/11.2.0/dbs/snap_orcl1.f';
new RMAN configuration parameters are successfully stored

after that we can delete obsolete files with command:

RMAN> delete obsolete;

good by :)