PING[ARC1]: Heartbeat failed to connect to standby ‘dgp’. Error is 1031.

just for short, digged another “circumstance” of oracle operation, in a data guard environment this time. i usually set up standby databases from a rman backup set, clone and adapt the spfile, clone the password file, create a new instance and so on. this works great, no problems so far.

this afternoon however, i got notified that some alert.log of a primary database is packed with messages of the following pattern:

Fri Aug 27 14:33:27 2010
Error 1031 received logging on to the standby
Fri Aug 27 14:33:27 2010
Errors in file e:\oracle\product\10.2.0\admin\dgs\bdump\dgs_arc0_1964.trc:
ORA-01031: insufficient privileges
FAL[server, ARC0]: Error 1031 creating remote archivelog file 'dgp'
FAL[server, ARC0]: FAL archive failed, see trace file.
Fri Aug 27 14:33:27 2010
Errors in file e:\oracle\product\10.2.0\admin\dgs\bdump\dgs_arc0_1964.trc:
ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Fri Aug 27 14:34:21 2010
Shutting down archive processes
Fri Aug 27 14:34:26 2010
ARCH shutting down
ARC2: Archival stopped
Fri Aug 27 14:34:26 2010
Error 1031 received logging on to the standby
Fri Aug 27 14:34:26 2010
Errors in file e:\oracle\product\10.2.0\admin\dgs\bdump\dgs_arc0_1964.trc:
ORA-01031: insufficient privileges
PING[ARC0]: Heartbeat failed to connect to standby 'dgp'. Error is 1031.

looks as if the fal (fetch archive log) service is somehow unable to establish a proper connection between primary and standby. furthermore, ora-01031 usually appears when some sysdba session failes to authenticate (remember the typical os authentication trap with windows, when a sysdba account login failes for an oracle service). so why can sys@primary no longer log in to sys@standby as sysdba?

actually simple. the password files, for external authentication, have become different, for another reason. i fact two different passwords for the sys accounts were active that way. take care, you need to test establishing sysdba sessions from a remote host to get this idea at all, because sys passwords on the server are ignored anyway.

have fun