Exadata after disk replacement asm disk remain mount_status closed header_status candidate

One of physical disk on exadata cell was droped for replacement status, after engineer change physical disk with new disk in v$asm_disk view shown that mount_status is closed and header_status is candidate:

select g.NAME,
       d.disk_number dsk_num,
       d.mount_status,
       d.header_status,
       d.OS_MB,
       d.path
 from v$asm_disk d, v$asm_diskgroup g
 where
      d.GROUP_NUMBER=g.GROUP_NUMBER
      and d.disk_number = 26;
 

NAME           DSK_NUM MOUNT_STATUS HEADER_STATUS      OS_MB PATH
----------- ---------- ------------ ------------- ---------- ------------------------------------------------
RECO           26      CACHED       MEMBER            582080  o/192.168.10.17;192.168.10.18/RECO_FD_02_x81cel04
DATA           26      CLOSED       CANDIDATE         5217280 o/192.168.10.13;192.168.10.14/DATA_FD_02_x81cel02

Solution : The problem will be resolved when the disks are added to the relevant disk groups by using the PATH value in the above query.

Processes must be performed by connecting to the ASM instance as SYSASM.

First of all check if rebalnce process is not working and if not than add disk manually.

check asm rebalance is not working:

sqlplus / as sysasm

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Aug 4 16:21:21 2023
Version 19.17.0.0.0

Copyright (c) 1982, 2022, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.17.0.0.0

SQL> select * from v$asm_operation;

no rows selected

SQL>

add disk manually:

alter diskgroup DATA add disk 'o/192.168.10.13;192.168.10.14/DATA_FD_02_x81cel02' rebalance power 4;

you can check progress of rebalance againg in v$asm_operation.

After rebalance process finish its job disk statuses must be cached and member.

 

select g.NAME,
       d.disk_number dsk_num,
       d.mount_status,
       d.header_status,
       d.OS_MB,
       d.path
 from v$asm_disk d, v$asm_diskgroup g
 where
      d.GROUP_NUMBER=g.GROUP_NUMBER
      and d.disk_number = 26;
 

NAME           DSK_NUM MOUNT_STATUS HEADER_STATUS      OS_MB PATH
----------- ---------- ------------ ------------- ---------- ------------------------------------------------
RECO           26      CACHED       MEMBER            582080  o/192.168.10.17;192.168.10.18/RECO_FD_02_x81cel04
DATA           26      CACHED       MEMBER            5217280 o/192.168.10.13;192.168.10.14/DATA_FD_02_x81cel02

During Installation ASM disks not shown

During installation Oracle Automatic Storage - ASM, I choose "Configure Oracle Grid Infrastructure for a Standalone Server (Oracle Restart) and click next

on next window appears that my disk for ASM is not shown

Solution:

click on "Change Discovery Path" buton
and write there "/dev/oracleasm/disks".

after that disk is shown and you can continue your installation/configuration.

Disk "DATA01" defines an unmarked device Dropping disk: failed

Hi,

Oracleasm deletedisk failed with error:

/usr/sbin/oracleasm deletedisk DATA01
Disk "DATA01" defines an unmarked device
Dropping disk: failed
Unable to delete disk "DATA01"

Here I'll show you step by step how to solve this problem :

/usr/sbin/oracleasm deletedisk DATA01
Disk "DATA01" defines an unmarked device
Dropping disk: failed
Unable to delete disk "DATA01"
-bash-3.2$ /usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Cleaning disk "DATA01"
Scanning system for ASM disks...
-bash-3.2$ /usr/sbin/oracleasm deletedisk DATA01
Disk "DATA01" defines an unmarked device
Dropping disk: failed
Unable to delete disk "DATA01"
-bash-3.2$ /etc/init.d/oracleasm scandisks
-bash: /etc/init.d/oracleasm: Permission denied
-bash-3.2$ 
-bash-3.2$ 
-bash-3.2$ 
-bash-3.2$ id
uid=501(grid) gid=1000(oinstall) groups=1000(oinstall),1200(asmadmin),1300(asmdba),1301(asmoper)
-bash-3.2$ 
-bash-3.2$ 
-bash-3.2$ logout
[root@test-db ~]# 
[root@test-db ~]# 
[root@test-db ~]# 
[root@test-db ~]# 
[root@test-db ~]# 
[root@test-db ~]# 
[root@test-db ~]# 
[root@test-db ~]# 
[root@test-db ~]# 
[root@test-db ~]# /usr/sbin/oracleasm listdisks
DATA01
DATA02
[root@test-db ~]# 
[root@test-db ~]# 
[root@test-db ~]# 
[root@test-db ~]# /usr/sbin/oracleasm deletedisk DATA01
Disk "DATA01" defines an unmarked device
Dropping disk: done
[root@test-db ~]# /usr/sbin/oracleasm listdisks
DATA02
[root@test-db ~]# /usr/sbin/oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
[root@test-db ~]# /usr/sbin/oracleasm listdisks
DATA02

Problem was that I tried to delete disk with grid user and it doesn't have permission to do that, therefore delete must use with root user.

CRS-4124: Oracle High Availability Services startup failed - ohasd failed to start: Inappropriate ioctl for device on Linux 6

During Automatic storage Management (ASM) installation occurred error

CRS-4124: Oracle High Availability Services startup failed 
ohasd failed to start: Inappropriate ioctl for device

Environment : 
OS : CentOS release 6.7

Grid : Oracle Grid Infrastructure 11g Release 2 (11.2.0.1.0) for Linux x86

This happened because in Linux 6 is not supported Oracle 11g Release 2 (11.2.0.1.0) 

Solution :  Here I''ll show you how to solve this problem.

1) open $GRID_HOME/crs/install/s_crsconfig_lib.pm file

2)Add the following lines before the # Start OHASD

my $UPSTART_OHASD_SERVICE = "oracle-ohasd";
my $INITCTL = "/sbin/initctl";

($status, @output) = system_cmd_capture ("$INITCTL start $UPSTART_OHASD_SERVICE");
if (0 != $status)
{
error ("Failed to start $UPSTART_OHASD_SERVICE, error: $!");
return $FAILED;
}
# Start OHASD

vi s_crsconfig_lib.pm

/Start OHASD  --- for search string in vi

3) create file with /etc/init/oracle-ohasd.conf file with below entries
# Oracle OHASD start
start on runlevel [35]
stop on runlevel [!35]
respawn
exec /etc/init.d/init.ohasd run >/dev/null 2>&1

4) De configuration of Oracle Restart stack with root user : 
[root@testdb install]# cd $GRID_HOME/crs/install/
[root@testdb install]# ./roothas.pl -deconfig -force -verbose
Checking for super user privileges
User has super user privileges
Parsing the host name
Using configuration parameter file: ./crsconfig_params
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Stop failed, or completed with errors.
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Delete failed, or completed with errors.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed 

resources on 'testdb'

CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'testdb'
CRS-2673: Attempting to stop 'ora.evmd' on 'testdb'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'testdb'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'testdb'
CRS-2677: Stop of 'ora.cssdmonitor' on 'testdb' succeeded
CRS-2677: Stop of 'ora.evmd' on 'testdb' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'testdb' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'testdb'
CRS-2677: Stop of 'ora.mdnsd' on 'testdb' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'testdb' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources 

on 'testdb' has completed
CRS-4133: Oracle High Availability Services has been stopped.
Successfully deconfigured Oracle Restart stack

5) run again root.sh script with root user and make overwrite "Y"

[root@testdb ~]# cd $GRID_HOME/
[root@testdb 11.2.0]# ./root.sh

................

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y

...............

That's all OHASD service started and working.

How to find mapping of ASM disks to Physical Devices?

Here I'll show you easy way to find which physical device is corresponding for ASM disk.
Here is Video of these procedures - Mapping of ASM disks to physical disks
Login with root user
1) Identify asm disks

/etc/init.d/oracleasm listdisks
DATA01

2) Query disks FOR DATA01

/etc/init.d/oracleasm querydisk -p DATA01
Disk "DATA01" is a valid ASM disk
/dev/sde2: LABEL="DATA01" TYPE="oracleasm"

or find this way

/etc/init.d/oracleasm querydisk -d DATA01
Disk "DATA01" is a valid ASM disk on device /dev/sde2[8,66]
ls -l /dev | grep 8 | grep 66
brw-r----- 1 root disk     8,  66 Jan 11 06:51 sde2

It means that physical partition sde2 is using for ASM disk DATA01.

ASM is not working after change hostname

On my test environment I changed hostname of Linux machine and after restart ASM did not start.
environment:

OS : Oracle Linux 6.5
DB : Oracle database 11g(11.2.0.3)

so let's simulate changing hostname and see results:

[root@oel6 ~]# hostname
oel6
[root@oel6 ~]# vi /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=testdb
[root@oel6 ~]# vi /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.220.128 testdb

[root@oel6 ~]# reboot

our hostname changed after restart we can check it and try to start ASM instance.
old hostname --- oel6 new hostname --- testdb

[root@testdb ~]# hostname
testdb
su - grid
-bash-4.1$ srvctl start asm
PRCR-1070 : Failed to check if resource ora.asm is registered
Cannot communicate with crsd

sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Thu Apr 2 14:05:38 2015

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup;
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service

Solution:
ok, ASM will not start because we need to reconfigure HAS(High Availability Service).
with root user go to $GRID_HOME/crs/install and run roothas.pl script for deconfiguration.

cd /u01/app/grid/product/11.2.0/grid/crs/install/
./roothas.pl -deconfig -force

Using configuration parameter file: ./crsconfig_params
CRS-4639: Could not contact Oracle High Availability Services
CRS-4000: Command Stop failed, or completed with errors.
CRS-4639: Could not contact Oracle High Availability Services
CRS-4000: Command Delete failed, or completed with errors.
CLSU-00100: Operating System function: opendir failed with error data: 2
CLSU-00101: Operating System error message: No such file or directory
CLSU-00103: error location: scrsearch1
CLSU-00104: additional error information: cant open scr home dir scls_scr_getval
CRS-4544: Unable to connect to OHAS
CRS-4000: Command Stop failed, or completed with errors.
Successfully deconfigured Oracle Restart stack

with root user go to $GRID_HOME/crs/install directory and run the roothas.pl script for reconfiguration

./roothas.pl

Using configuration parameter file: ./crsconfig_params
LOCAL ADD MODE
Creating OCR keys for user 'grid', privgrp 'oinstall'..
Operation successful.
LOCAL ONLY MODE
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4664: Node testdb successfully pinned.
Adding Clusterware entries to upstart

testdb     2015/04/02 14:29:27     
/u01/app/grid/product/11.2.0/grid/cdata/testdb/backup_20150402_142927.olr
Successfully configured Oracle Grid Infrastructure for a Standalone Server

with grid user Register and start the resources

crsctl start resource ora.cssd
CRS-2672: Attempting to start 'ora.cssd' on 'testdb'
CRS-2672: Attempting to start 'ora.diskmon' on 'testdb'
CRS-2676: Start of 'ora.diskmon' on 'testdb' succeeded
CRS-2676: Start of 'ora.cssd' on 'testdb' succeeded

now we can add the listener as a resource and after that we can start it

srvctl add listener -l LISTENER
srvctl start listener
srvctl status listener
Listener LISTENER is enabled
Listener LISTENER is running on node(s): testdb

now we can add the asm we do not need to create spfile for asm. we can use the one before.
if you haven't spfile you can simple create pfile and then create spfile from this pfile for example create pfile - init+ASM.ora in $GRID_HOME/dbs with parameters:

instance_type=ASM
asm_diskstring='/dev/oracleasm/disks/DATA01'
asm_diskgroups='DG01'

sqlplus / as sysasm
SQL> startup pfile=init+ASM.ora;
SQL> create spfile='+DG01' from pfile='init+ASM.ora';
SQL> shutdown immediate;

with grid user

srvctl add asm -l LISTENER -p "/u01/app/grid/product/11.2.0/grid/dbs/init+ASM.ora"
srvctl start asm
srvctl status asm
ASM is running on testdb

at last we can add database

srvctl add database -d testdb -o /u01/app/oracle/product/11.2.0/dbhome_1 -p 
+DG01/spfiletestdb.ora -r PRIMARY -s open -t immediate -a "DG01"
srvctl start database -d testdb
srvctl status database -d testdb
Database is running.

that's all, now we have new hostname and asm is up and running.

srvctl status asm
ASM is running on testdb

Moving an Oracle Database from filesystem to ASM

Here i'l show you how to migrate database from filesystem to ASM(Automatic Storage Management) on same host.
environment

OS : Oracle Linux 6.5
DB : Oracle database 11g(11.2.0.3.0)

First of all we need to configure Grid infrastructure, with root user make next steps:
1) install ASM packages, which you can find on official oracle site, for example:
HERE
2) add disk for ASM and partition it
3) add separate Operating system user and groups for ASM, which is recommendation from oracle, for example: grid user and groups: asmadmin,asmdba,asmoper

groupadd  -g 2100 asmadmin
groupadd  -g 2200 asmdba
groupadd  -g 2300 asmoper

mkdir -p /u01/app/grid
mkdir -p /home/grid
useradd -s /bin/bash -d /home/grid  -g oinstall -G asmadmin,asmdba,asmoper grid

4) give permissions to grid user

chown -R grid:oinstall /u01/app/grid
chown -R grid:oinstall /home/grid
chmod -R 775 /u01/app/grid
passwd grid

5) configure ASM library driver

/etc/init.d/oracleasm configure
grid
asmadmin
y
y

6) Mark disk for ASM

/etc/init.d/oracleasm createdisk DATA01 /dev/sdc1

7) add grid user limits in /etc/security/limits.conf

vi /etc/security/limits.conf

grid   soft   nofile    1024
grid   hard   nofile    65536
grid   soft   nproc    16384
grid   hard   nproc    16384
grid   soft   stack    10240
grid   hard   stack    32768

8) install and configure Grid infrastructure with grid user

Now we are ready for migrate database from filesystem to ASM.
with oracle OS user:
1) connect to database with rman and make copy for database to the ASM

rman target /

RMAN> backup as copy database format '+DG01'; --our diskgroup name

2) make copy for current controlfile to ASM

backup as copy current controlfile format '+DG01';

3) Restore your spfile to a location inside ASM with the following rman script.

run {
backup as backupset spfile;
restore spfile to '+DG01/spfiletestdb.ora';
}

4) create pfile in $ORACLE_HOME/dbs with parameter SPFILE='+DG01/spfiletestdb.ora' and delete spfile from $ORACLE_HOME/dbs

cd $ORACLE_HOME/dbs
vi inittestdb.ora
SPFILE='+DG01/spfiletestdb.ora'

rm -rf spfiletestdb.ora

5) next steps include restore controlfile, switch database to copy, swith tempfile, add new redo logs in ASM and drop olders from filesystem:

sqlplus / as sysdba
SQL> startup nomount;
SQL> alter system set control_files='+DG01' scope=spfile;
SQL> shutdown immediate;
SQL> startup nomount;
SQL> ! rman target /
RMAN> restore controlfile from '+dg01/testdb/controlfile/Backup.263.875890551';
RMAN> alter database mount;
RMAN> switch database to copy;
RMAN> recover database;
RMAN> run {
set newname for tempfile 1 to '+DG01';
switch tempfile 1;
}
RMAN> exit
SQL> alter database add logfile '+DG01' size 50M;
SQL> alter database add logfile '+DG01' size 50M;
SQL> alter database add logfile '+DG01' size 50M;
SQL> alter database open resetlogs;
SQL> alter database drop logfile '/u01/app/oracle/oradata/TESTDB/onlinelog/redo03.log';
SQL> alter database drop logfile '/u01/app/oracle/oradata/TESTDB/onlinelog/redo02.log';
SQL> alter database drop logfile '/u01/app/oracle/oradata/TESTDB/onlinelog/redo01.log';

if you encounter an ORA-01623 while trying to delete a member just switch logs and retry. To switch a log execute

alter system switch logfile;

That's all our database completely migrated to the ASM.

Duplicate database from ASM to non ASM Database using RMAN

Operating system --- Oracle Linux 6
Database --- 11G R2
Today i show you how to duplicate database which is in ASM to non ASM on different host.
Target --- database with ASM which is used to create another database on another host.
Destination --- database which located on file system will be created from target.

For duplication there is two method: from backup and from active database, here is described active database duplication.

On destination already installed Oracle Software.
1) transfer password file from target to destination.
2) on destination register listener with static registration.
3) on Target, create pfile, transfer it to destination host and modify next parameters:

*.audit_file_dest =/u01/app/oracle/testdup/adump
*.db_name ="testdup"
*.instance_name =testdup
*.control_files =/u01/app/oracle/testdup/control01.ctl
*.db_file_name_convert =("+DATA","/u01/app/oracle/testdup")
*.log_file_name_convert =("+DATA","/u01/app/oracle/testdup")
*.undo_management =AUTO
*.undo_tablespace='UNDOTBS1'
*.db_block_size=8192

4) on destination, create corresponding directories with oracle O/S user.
5) export new sid and start instance;

export ORACLE_SID=testdup
sqlplus / as sysdba
startup nomount pfile=/u01/app/oracle/product/11.2.0/dbhome_1/dbs/inittestdup.ora;

6) on destination:

rman target sys/password@orcl auxiliary sys/password@testdup
duplicate database to testdup from active database;

Possible Errors:
You can get some errors , cann't restore datafile & onlineredo log files, in this case you need to create directory ,It should be indicate.

testdup/datafile
testdup/onlinelog

That's all, good luck.

CRS-0184: Cannot communicate with the CRS daemon CRS-4639: Could not contact Oracle High Availability Services

After restarted server and I was starting ASM instance appear error :

CRS-0184: Cannot communicate with the CRS daemon 
CRS-4639: Could notcontact Oracle High Availability Services

I will show you how to solve this, there may be many reasons I'll write here which is very often.
First of all check services and if they aren't started try to start:

crs_stat -t
CRS-4639: Could not contact Oracle High Availability Services

CRS-4000: Command Status failed, or completed with errors.

crsctl start resource ora.cssd
 

If there still error then try loging with root user and remove all files under /tmp/.oracle or /var/tmp/.oracle, because sometimes when CRS server reboot it try to create sockets under /tmp/.oracle or /var/tmp/.oracle and there are already previous socket files...which are protecting to create new sockets. restart CRS on faulty node or even reboot that machine!.

if it is not enough then check $ORACLE_HOME/bin/oracle executable file permissions it must have:

[oracle@oel5 ~]$ ls -lrt $ORACLE_HOME/bin/oracle
-rwsr-s--x 1 oracle dba 210824714 May 21  2012 /u0/app/oracle/product/11.2.0/dbhome_1/bin/oracle

If it is not same permission as shown above change it with:

chmod 6751 $ORACLE_HOME/bin/oracle

If there is still errors try

crsctl enable has
crsctl start has

If the errors still try with root

[root@oel5 ~]# /etc/init.d/syslog start
[root@oel5 ~]# /etc/init.d/ohasd start

I think it will be helpful for you,
thank you.

ORA-01078: failure in processing system parameters ORA-01565: error inidentifying file '+DATA/ORCL/spfileorcl.ora' ORA-17503: ksfdopn:2 Failed to open file +DATA/ORCL/spfileorcl.ora ORA-01034: ORACLE not available ORA-27123: unable to attach to shared memory segment Linux-x86_64 Error: 13: Permission denied

There are many things for occur error:

ORA-01078: failure in processing system parameters 
ORA-01565: error inidentifying file '+DATA/ORCL/spfileorcl.ora'; 
ORA-17503: ksfdopn:2 Failed to open file +DATA/ORCL/spfileorcl.ora 
ORA-01034: ORACLE not available 
ORA-27123: unable to attach to shared memory segment 
Linux-x86_64 Error: 13: Permission denied

I write here often issues and their solutions:
first of all check permission on $ORACLE_HOME/bin/oracle file in both ASM and RDBMS home, this file is oracle executable file, there must be permission 6751 for this file. let me check

ls -l $ORACLE_HOME/bin/oracle
-rwxrwxrwx 1 oracle asmadmin /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle

change it with root user

chmod 6751 /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle

another often issue is that oracle user have not proper groups in operating system level, for normal work there must be this groups

[oracle@node1]$ id
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(asmadmin),
54324(asmdba),54326(oper)

if they aren't then add oracle user in these groups with command

usermod -G usermod -G oinstall,dba,oper,asmdba,asmadmin oracle

if you have grid user also check for it, there must be groups shown below

[grid@node1]$ id
uid=54322(grid) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(asmadmin),
54324(asmdba),54325(asmoper)

i hope this will usefull for you, because i spent many time for solve this error.
thank you.

Change ASM disk group from normal redundancy to external redundancy

Hello,

When the disk group was created with normal redundancy, the only way to change is to drop the disk group and recreate a new one.
Warning: If you drop disk group your data will be lost.
1) shutdown database

shutdown immediate;

2) startup in mount mode

startup mount;

3) make database full backup, archivelog backup, spfile backup and controlfile backup

RMAN> backup device type disk format '/u02/backup/backup/database_backup%U' database;

RMAN> backup device type disk format '/u02/backup/backup/archivelog%U' archivelog all;

SQL>  create pfile='/u02/backup/initorcl.ora' from spfile;

SQL>  alter database backup controlfile to '/u02/backup/control.ctl';

4) shutdown database;

shutdown immediate;

5) login as sysasm and drop disk group

[oracle@oel6 backup]$ . oraenv
ORACLE_SID = [orcl] ? +ASM
sqlplus / as sysasm

SQL> drop diskgroup DISKS including contents;

SQL> shutdown immedaite;

SQL> startup nomount;

6) run asmca and create new disk group with external redundancy

SQL> select status from v$instance;

STATUS
------------
STARTED

7) ASM instance is already started, now we must restore database and all related files.

[oracle@oel6]$ . oraenv
ORACLE_SID = [+ASM] ? orcl
sqlplus / as sysdba

SQL> startup nomount pfile='/u02/backup/initorcl.ora';

SQL> create spfile='+DISKS' from pfile='/u02/backup/initorcl.ora';

SQL> !rman target /

RMAN> restore controlfile from '/u02/backup/control.ctl';

RMAN> alter database mount;

RMAN> restore database;

RMAN> recover database;

unable to find archived log
archived log thread=1 sequence=8
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 09/25/2013 12:11:31
RMAN-06054: media recovery requesting unknown archived log for thread 1 with sequence 8 
and starting SCN of 21904164

While recovery it will give an error for archive log missing, this is expected we need to open the database with resetlogs as

RMAN> alter database open resetlogs;

8)
for using correct spfile:

ASMCMD> cp '/u0/app/oracle/product/11.2.0/dbhome_1/dbs/spfileorcl.ora' '+DISKS/ORCL/spfileorcl.ora' 

delete spfile from file system

rm -rf /u0/app/oracle/product/11.2.0/dbhome_1/dbs/spfileorcl.ora

edit /u0/app/oracle/product/11.2.0/dbhome_1/dbs/initorcl.ora, which must contains link to our spfile in disk group.

*.SPFILE='+DISKS/orcl/spfileorcl.ora'

that's all.

Marking disk as an ASM disk FAILED

I ran this command and got the message that the disk does not exist or is not instantiated

/etc/init.d/oracleasm createdisk DISK1 /dev/sdb1
Marking disk "DISK1" as an ASM disk: [FAILED]

I checked for ASM create errors in /var/log/oracleasm and saw this ASM disk error message:

 Disk "DISK1" does not exist or is not instantiated
 Writing disk header: done
 Instantiating disk: oracleasm-instantiate-disk:
 Unable to create ASM disk "DISK1": Permission denied

Solution :
The ASM error - disk does not exist or is not instantiated can be caused by many issues but this is very frequent :
SELINUX isn't disabled.

vi /etc/sysconfig/selinux SELINUX=disabled

Then reboot your system and run getenforce to confirm that SELINUX is disabled:

root> /usr/sbin/getenforce Disable

that's all.

ORA-01078: failure in processing system parameters ORA-29701: unable to connect to Cluster Synchronization Service

startup of ASM instance failed with error:
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service

check services by command:

crs_stat -t
Name           Type           Target    State     Host        
------------------------------------------------------------
ora.DATA.dg    ora....up.type OFFLINE   OFFLINE      
ora....ER.lsnr ora....er.type ONLINE    ONLINE    oralnxt01   
ora....2C.lsnr ora....er.type ONLINE    ONLINE    oralnxt01   
ora.asm        ora.asm.type   OFFLINE   OFFLINE      
ora.cssd       ora.cssd.type  ONLINE    ONLINE    oralnxt01   
ora.diskmon    ora....on.type ONLINE    ONLINE    oralnxt01   
ora.orcl.db    ora....se.type OFFLINE   OFFLINE    

solution:
start services manually

crsctl start resource ora.cssd
CRS-2672: Attempting to start 'ora.cssd' on 'oralnxt01'
CRS-2679: Attempting to clean 'ora.diskmon' on 'oralnxt01'
CRS-2681: Clean of 'ora.diskmon' on 'oralnxt01' succeeded
CRS-2672: Attempting to start 'ora.diskmon' on 'oralnxt01'
CRS-2676: Start of 'ora.diskmon' on 'oralnxt01' succeeded
CRS-2676: Start of 'ora.cssd' on 'oralnxt01' succeeded

sqlplus / as sysasm

SQL*Plus: Release 11.2.0.1.0 Production on Fri Jan 25 10:25:56 2013

Copyright (c) 1982, 2009, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup;
ASM instance started

Total System Global Area  283930624 bytes
Fixed Size                  2212656 bytes
Variable Size             256552144 bytes
ASM Cache                  25165824 bytes
ASM diskgroups mounted

that's all. enjoy :)

Could not validate asmsnmp password due to following error ora-01031:insufficient privileges

During Creation new database with dbca, It shows following errors
"could not validate asmsnmp password due to following error ora-01031:insufficent privileges"

very strange, in asm instance alert log found message

ORA-01918: user 'ASMSNMP' does not exist

somehow asmsnmp user is not created during grid infrastructure installation, so we have to cretae it manually.

solution :
1) create password file for asm instance.
orapwd file=/u0/app/11.2.0/grid/dbs/orapw+ASM password=anypassword
2) sqlplus / as sysasm
create user asmsnmp identified by password;
grant sysdba to asmsnmp;

enjoy with database creation.