RAC インスタンスの 1 つで、頻繁に以下のエラーが発生します。同じエラーが原因で、アプリケーションがハングアップし、問題が発生しています。
Sat Aug 04 06:57:26 2012
IPC Send timeout detected.Sender: ospid 28201
Receiver: inst 1 binc 8 ospid 3537
GATHER_STATS_JOB encountered errors. Check the trace file.
Errors in file /opt/oracle/db/diag/rdbms/mdspdb/mdspdb5/trace/mdspdb5_j001_28201.trc:
ORA-12805: parallel query server died unexpectedly
ここでも、次のようなエラーがもう 1 つあります。
Thu Aug 09 03:43:05 2012
IPC Send timeout detected.Sender: ospid 18645
Receiver: inst 1 binc 8 ospid 3537
GATHER_STATS_JOB encountered errors. Check the trace file.
Errors in file /opt/oracle/db/diag/rdbms/mdspdb/mdspdb5/trace/mdspdb5_j002_18645.trc:
ORA-20011: Approximate NDV failed: ORA-12805: parallel query server died unexpectedly
トレース ファイルからの出力は次のとおりです。
Trace file /opt/oracle/db/diag/rdbms/mdspdb/mdspdb5/trace/mdspdb5_j001_22631.trc
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
ORACLE_HOME = /opt/oracle/db/product/11.1.0/db_1
System name: Linux
Node name: mdspdb05
Release: 2.6.16.46-0.12-smp
Version: #1 SMP Thu May 17 14:00:09 UTC 2007
Machine: x86_64
Instance name: mdspdb5
Redo thread mounted by this instance: 5
Oracle process number: 107
Unix process pid: 22631, image: oracle@mdspdb05 (J001)
*** 2012-08-15 03:43:12.742
*** SESSION ID:(486.34904) 2012-08-15 03:43:12.742
*** CLIENT ID:() 2012-08-15 03:43:12.742
*** SERVICE NAME:(SYS$USERS) 2012-08-15 03:43:12.742
*** MODULE NAME:(DBMS_SCHEDULER) 2012-08-15 03:43:12.742
*** ACTION NAME:(ORA$AT_OS_OPT_SY_40411) 2012-08-15 03:43:12.742
IPC Send timeout detected.Sender: ospid 22631
Receiver: inst 1 binc 8 ospid 3537
SKGXP: SKGXPCTX: 0x0x2abb52d24a38 ctx
SKGXP:
SKGXP: WAIT HISTORY
SKGXP: Time(msec) Wait Type Return Code
SKGXP: ---------- --------- ------------
SKGXP: 2501 NORMAL TIMEDOUT
SKGXP: 0 NORMAL TIMEDOUT
SKGXP: 0 NORMAL TIMEDOUT
SKGXP: 2500 NORMAL TIMEDOUT
SKGXP: 0 NORMAL TIMEDOUT
SKGXP: 0 NORMAL TIMEDOUT
SKGXP: 2503 NORMAL TIMEDOUT
SKGXP: 0 NORMAL TIMEDOUT
SKGXP: 0 NORMAL TIMEDOUT
SKGXP: 2501 NORMAL TIMEDOUT
SKGXP: 0 NORMAL TIMEDOUT
SKGXP: 0 NORMAL TIMEDOUT
SKGXP: 2503 NORMAL TIMEDOUT
SKGXP: 0 NORMAL TIMEDOUT
SKGXP: 0 NORMAL TIMEDOUT
SKGXP: 2500 NORMAL TIMEDOUT
SKGXP: wait delta 61 sec (61423 msec) ctx ts 0x79e31f last ts 0x79e31f
SKGXP: user cpu time since last wait 0 sec 0 ticks
SKGXP: system cpu time since last wait 0 sec 0 ticks
SKGXP: locked 1
SKGXP: blocked 25048
SKGXP: timed wait receives 15
SKGXP: admno 0x66859bfb admport:
SKGXP: SSKGXPT 0x52d2597c flags sockno 18 IP 192.168.1.10 UDP 61826
SKGXP: context timestamp 0x79e31f
SKGXP: buffers queued on port 0x2abb53449f50
SKGXP:
SKGXP: Dumping Connection Handle Table
SKGXP: sconno accono ertt state seq# RcvPid TotCreditsSKGXP: sent rtrans acks
SKGXP: CNH Table Bucket: 0
SKGXP: 0x03a8c901 0x50790ec9 46982647250960 46982647250948 46982647284070 46982647267591 46982647250952SKGXP: 363d 1d 33126d
SKGXP: CNH Table Bucket: 1
SKGXP: 0x03a8c902 0x051d84d8 16 46982647250948 46982647284112 46982647259415 46982647250952SKGXP: 405d 0d 33168d
SKGXP: CNH Table Bucket: 2
SKGXP: 0x03a8c903 0x51d441b2 16 46982647250948 46982647284217 46982647258187 46982647250952SKGXP: 510d 0d 33273d
SKGXP: CNH Table Bucket: 3
SKGXP: 0x03a8c904 0x7d38c1af 16 46982647250948 46982647283952 46982647267615 46982647250952SKGXP: 245d 0d 33008d
SKGXP: CNH Table Bucket: 4
SKGXP: 0x03a8c905 0x22782e02 16 46982647250948 46982647284097 46982647283080 46982647250952SKGXP: 390d 0d 33153d
SKGXP: CNH Table Bucket: 5
SKGXP: 0x03a8c906 0x19e5dbdb 16 46982647250948 46982647284141 46982647283122 46982647250952SKGXP: 434d 0d 33197d
SKGXP: CNH Table Bucket: 6
SKGXP: 0x03a8c907 0x024222c8 16 46982647250948 46982647283839 46982647258304 46982647250952SKGXP: 132d 0d 32895d
SKGXP: CNH Table Bucket: 7
SKGXP: 0x03a8c908 0x1f3d5569 16 46982647250948 46982647284073 46982647283097 46982647250952SKGXP: 366d 0d 33129d
SKGXP: CNH Table Bucket: 8
SKGXP: 0x03a8c909 0x4b4e08b3 16 46982647250948 46982647283990 46982647267605 46982647250952SKGXP: 283d 1d 33046d
SKGXP: CNH Table Bucket: 9
SKGXP: 0x03a8c90a 0x32e47d79 16 46982647250948 46982647283856 46982647258242 46982647250952SKGXP: 149d 0d 32912d
SKGXP: CNH Table Bucket: 10
SKGXP: 0x03a8c90b 0x60420275 16 46982647250948 46982647283829 46982647259471 46982647250952SKGXP: 122d 0d 32885d
SKGXP: CNH Table Bucket: 11
SKGXP: 0x03a8c90c 0x5521db22 16 46982647250948 46982647283834 46982647259493 46982647250952SKGXP: 127d 0d 32890d
SKGXP: CNH Table Bucket: 12
SKGXP: 0x03a8c90d 0x1696da78 16 46982647250948 46982647283708 46982647254481 46982647250952SKGXP: 1d 373d 0d
SKGXP: slot 3 rqh=0x2abb52d2e0e8 seq=32763 len=2144 accno=0x1696da78 start TS=0x2f31c rt TS=0x79e56c X CNT=374
SKGXP:
SKGXP: Dumping Accept Handle Table
SKGXP: ach accono sconno admno state SndPid seq# rcv rtrans acks credits
Dumping KSXP Connection Handle: cnh: 0x2abb54b97098
Link in Component Context : 0x2abb52fc0c08 <- cnh -> 0x2abb52fc0c08
Link in Connection Cache : 0x2abb53070050 <- cnh -> 0x2abb52f68158
Pending Async Sends : First: 0x2abb52d2e030 Last 0x2abb52d2e030
Name:
State: 1 ==> OPEN
Flags: 0x0 ==>
Connect request handle: (nil)
Miners Canary : (nil)
OSD Connection handle : 0x2abb55286938
Stats pointer : 0x19ee7f470
System incarnation when connection was made: 78
Dumping Target id for cnh
vers: 1 type: (1 ==> KSXP_TRANSPORT_ID_SKGXPPID) inst: 1 ptid: 401 ukey: 8
Protocol Info: vendor: 1 proto: 2 maj: 3 min: 0
pnum: 91 ospid: 3537
p1_pad1: 0 p2_pad1: (nil)
pad2: 0 pad3: 0
Dumping SKGXP Port ID: 0x2abb54b97120
SKGXP: SKGXPID 0x2abb54b97120
SKGXP: skgxp port number 0x25a32042 process id 3537
SKGXP: admin port id
SKGXP: SKGXPGPID 0x7fff596feb58
SKGXP: Internet address 192.168.1.12
SKGXP: UDP port number 28958
Dumping connection stats 0x19ee7f470
Dumping SKGXP Connection Handle: osd cnh: 0x2abb55286938
SKGXP: SKGXPCNH: 0x2abb55286938 SKGXPCON_OPEN (2) sconno 3a8c90d accono 1696da78 admno fd8255e
SKGXP: Remote admin port
SKGXP: SSKGXPT 0x55286974 flags SSKGXPT_WRITE sockno 18 IP 192.168.1.12 UDP 28958
SKGXP: Remote data port
SKGXP: SSKGXPT 0x55286a08 flags SSKGXPT_WRITE sockno 18 IP 192.168.1.12 UDP 29092
SKGXP: no disconnect notification request handle
SKGXP: next seqno 32764 credits 7 ertt 16 resends on con 488510
*** 2012-08-15 03:43:12.792
Submitting synchronized dump request [268435460]
*** 2012-08-15 03:43:14.796
kxfp_send_callback
Send timed out to slave 0 inst 1 (qref 0x19e82bf88)
kxfpgsg
Error 12805 encountered
*** 2012-08-15 03:43:14.831
ORA-20011: Approximate NDV failed: ORA-12805: parallel query server died unexpectedly
*** 2012-08-15 03:43:14.831
GATHER_STATS_JOB: GATHER_TABLE_STATS('"IMUSE01"','"SUBSCRIBEINFO"','""', ...)
ORA-20011: Approximate NDV failed: ORA-12805: parallel query server died unexpectedly
オラクルのバージョン:オラクル 11.1.0.6.0 RAC