November 27, 2012

November 5, 2012

DEBUG RMAN: "RMAN-20052: invalid datafile create SCN" due to PLUGIN_SCN out of SYNC

Below error persists for one database during SYNC operation:

RMAN> resync catalog;

starting full resync of recovery catalog
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of resync command on default channel at 11/05/2012 07:10:03
RMAN-20052: invalid datafile create SCN

Target DB version:
11.2.0.2.6

Catalog DB version:
11.2.0.2.6

Catalog schema version:
catalog > Select * from rcver ;
VERSION
------------
11.02.00.03


After compare between target and catalog, found the create_scn for all datafiles and tempfiles are SYNC in both DB:
CATALOG DB:
catalog > select file#,CREATION_CHANGE# from RC_DATAFILE where DB_NAME='ATPPSS' order by 1;
     FILE# CREATION_CHANGE#
---------- ----------------
         1                7
         2             2178
         3           969548
         4            18151
         5          1002153
         6          1053956
         7          1054333
         8          1501572
         9          1502015
        10          1506616
        11          1515561
        12          1516921
        13          5361004
        14          5779799
14 rows selected.

TARGET DB:
target > select file#,CREATION_CHANGE# from v$datafile;
     FILE# CREATION_CHANGE#
---------- ----------------
         1                7
         2             2178
         3           969548
         4            18151
         5          1002153
         6          1053956
         7          1054333
         8          1501572
         9          1502015
        10          1506616
        11          1515561
        12          1516921
        13          5361004
        14          5779799
14 rows selected.

After debug rman session, we can see the error report when doing DBMS_RCVCAT.CHECKDATAFILE on file# 13:
DBGRPC:        krmxrpc - channel default kpurpc2 err=0 db=rcvcat proc=RMANADMIN.DBMS_RCVCAT.CHECKDATAFILE excl: 0
   DBGRCVCAT: checkDatafile - size changed for file#: 12 from 131072 to 153600
   DBGRCVCAT: incResynActions - Not debugging
   DBGRCVCAT: setCloneName: file#=12, creation_fname=1516921, plugin_change#=0, old_clone_fname=, new_clone_fname=NONE
DBGRESYNC:     channel default: Resyncing datafile +DATA/atppss/datafile/example.3079.785280031 [23:38:41.110] (resync)
DBGRESYNC:     channel default: Calling checkDataFile for fileno 13 size 131072 [23:38:41.110] (resync)
DBGRPC:        krmxrpc - channel default kpurpc2 err=20052 db=rcvcat proc=RMANADMIN.DBMS_RCVCAT.CHECKDATAFILE excl: 97
DBGRPC:        krmxrpc - channel default kpurpc2 err=0 db=rcvcat proc=RMANADMIN.DBMS_RCVCAT.CANCELCKPT excl: 0
   DBGRCVCAT: cancelCkpt - rollback, released all locks
DBGRPC:        krmxrpc - channel default kpurpc2 err=0 db=target proc=SYS.DBMS_BACKUP_RESTORE.CFILEUSECURRENT excl: 0

Below is the defination for 20052 in DBMS_RCVCAT.CHECKDATAFILE:
PROCEDURE CHECKDATAFILE()
IF (plugged_readonly = 'NO' AND create_scn > this_ckp_scn) THEN              
   raise_application_error(-20052, 'Invalid datafile create SCN');           
ELSIF (plugged_readonly = 'YES' AND plugin_scn > this_ckp_scn) THEN          
   raise_application_error(-20055, 'Invalid datafile plugin SCN');           
END IF;                                                                      
...........
   IF (create_scn = dfRec.create_scn AND                                     
       plugin_scn = dfRec.plugin_scn) THEN                                   
       ...........
   ELSIF ((case when plugged_readonly = 'NO' then                            
           create_scn else plugin_scn end) >                                 
          (case when dfRec.plugged_readonly = 'NO' then                      
           dfRec.create_scn else dfRec.plugin_scn end)) THEN
        ...........                 
   ELSE -- (create_scn < dfRec.create_scn)                                   
      -- The client passed us a create SCN for this datafile that is         
      -- less than the SCN in the rcvcat.  I.e., the target database         
      -- controlfile now contains a previous incarnation of this datafile.   
      -- This can happen only if the user has some old controlfile, or       
      -- has done a resetlogs and not told us about it.                      
      IF (plugged_readonly = 'NO') THEN                                      
         raise_application_error(-20052, 'Invalid datafile create SCN');     
      ELSE                                                                   
         raise_application_error(-20055, 'Invalid datafile plugin SCN');     
      END IF;
    END IF;  

All the judgement done by three argument: plugin_scn, plugged_readonly,create_scn. Let's check above three values in both target DB and catalog DB:
CATALOG DB:
catalog > select CREATE_SCN,PLUGGED_READONLY,PLUGIN_SCN from df where DBINC_KEY='3199420' and FILE# =13;
CREATE_SCN PLU PLUGIN_SCN
---------- --- ----------
   5361004 NO      992188

TARGET DB:
target > select CREATION_CHANGE#,PLUGGED_READONLY,PLUGIN_CHANGE# from v$datafile where file#=13;
CREATION_CHANGE# PLU PLUGIN_CHANGE#
---------------- --- --------------
         5361004 NO               0

Now we can clearly see that red part's differ is the reason we got the error.
Since pluggable-DB is a new feature in 12c, and not published in any current version as yet.
So currently all databases’ all datafiles’ PLUGIN_CHANGE# should be 0.

The PLUGIN_SCN is 992188 in CATALOG DB is obviously not correct. Un-Register/Re-register the DB in catalog may or may not help to solve this issue.
More......