Domino Server Suddenly crash

HCL Domino Domino Forum
1

Hi,

I’m seeking help in finding out the reason why our Domino Server suddenly crash. Below is the detailed info from the our server.

Domino version: Release 8.5.3 September 15, 2011

OS Version: Windows/2008 R2 6.1 [64-bit]

Start Time: 03/28/2012 06:21:26 PM

Crash Time: 04/02/2012 06:19:22 PM

Uptime: 4 day(s) 23:57:56

Error message: PANIC: Process C:\Windows\system32\mmc.exe (1000/0x3E8) has terminated abnormally

Process: nserver

Callstack: [ 1] 0x7721135a ntdll.ZwWaitForSingleObject+10 (CCC8064E827;26b2749;4;4134fc0)

[ 2] 0x7FEFD3D10AC KERNELBASE.WaitForSingleObjectEx+156 (5;5;0;20b4)

@[ 3] 0x004b4cdb nnotes.FRSendCommandToService+1943 (5ab0b800;5ab0bb18;0;4b0fc4)

@[ 4] 0x004b85c4 nnotes.OSRunExternalScript+5860 (4;5;272a084;1)

@[ 5] 0x004b1df1 nnotes.FRTerminateWindowsResources+2277 (0;0;0;0)

@[ 6] 0x004bb58e nnotes.OSFaultCleanupExt+622 (0;4b118b;150014;272e1b0)

@[ 7] 0x004bc005 nnotes.OSFaultCleanup+29 (375746F4C5C3A44;C6F6E696D6F445C;E6F6D636F72706E;63231204558452E)

@[ 8] 0x004d597a nnotes.OSNTUnhandledExceptionFilter+626 (5ab0ea90;2711164;5ab0fb00;4d)

@[ 9] 0x00445a6c nnotes.Panic+856 (12137ab8;2713c9b;7e933028;3e8)

@[10] 0x10007905 nserverl.ProcessMonitorTask+681 (7af600f6;0;4f0;bdc0df4)

@[11] 0x10001b21 nserverl.Scheduler+969 (0;0;0;0)

@[12] 0x0045654e nnotes.ThreadWrapper+330 (0;0;0;0)

[13] 0x770b652d kernel32.BaseThreadInitThunk+13 (0;0;0;0)

[14] 0x771ec521 ntdll.RtlUserThreadStart+33 (0;0;0;0)

Open Databases: "

I’m new to Domino administration and would greatly help me if we can prevent this crash from happening again.

Thank you so much in advance.

Chris Valdez

2

Subject: Domino Server 8.5.3 FP1 - nserver: FATAL THREAD (Panic)

I wanted to let you know that we are seeing the same crash that your reproted

8.5.3FP1 (64-bit server)

Windows/2008 R2 6.1 (64-bit) (Build 7601), PlatID=2, Service Pack 1 (8 Processors)

PANIC: Process c:\Program Files\tivoli\tsm\domino\domdsmc.exe (7844/0x1EA4) has terminated abnormally

############################################################

thread 245/249: [ nserver: 10cc: 207c] FATAL THREAD (Panic)

FP=0x553c6188, PC=0x77c9135a, SP=0x553c6188

stkbase=0x553d0000, total stksize=4194304, used stksize=40568

EAX=0x00000000, EBX=0x00000000, ECX=0x00000000, EDX=0x00000000

ESI=0x000493e0, EDI=0x00004710, CS=0x00000033, SS=0x0000002b

DS=0x00000000, ES=0x00000000, FS=0x00000000, GS=0x00000000 Flags=0x00000287

############################################################

[ 1] 0x77c9135a ntdll.ZwWaitForSingleObject+10 (1D776B6D57B0,26b47b9,4,4137fc0)

[ 2] 0x7FEFE2310DC KERNELBASE.WaitForSingleObjectEx+156 (5,5,0,4710)

@[ 3] 0x004b584b nnotes.FRSendCommandToService+1943 (553cb800,553cbb18,0,4b1b34)

@[ 4] 0x004b9134 nnotes.OSRunExternalScript+5860 (4,5,272c0a4,1)

@[ 5] 0x004b2961 nnotes.FRTerminateWindowsResources+2277 (0,0,0,0)

@[ 6] 0x004bc0fe nnotes.OSFaultCleanupExt+622 (0,4b1cfb,150014,27301d0)

@[ 7] 0x004bcb75 nnotes.OSFaultCleanup+29 (375746F4C5C3A43,C6F6E696D6F445C,E6F6D636F72706E,3334204558452E)

@[ 8] 0x004d64ea nnotes.OSNTUnhandledExceptionFilter+626 (553cea90,2713184,553cfb00,4d)

@[ 9] 0x00445a60 nnotes.Panic+856 (14d4c478,2715cbb,d65329a0,1ea4)

@[10] 0x1000792d nserverl.ProcessMonitorTask+681 (d85401c0,0,10cc,be00df4)

@[11] 0x10001b21 nserverl.Scheduler+969 (0,0,0,0)

@[12] 0x00456572 nnotes.ThreadWrapper+330 (0,0,0,0)

[13] 0x776c652d kernel32.BaseThreadInitThunk+13 (0,0,0,0)

[14] 0x77c6c521 ntdll.RtlUserThreadStart+33 (0,0,0,0)

I would like to know if your using DAOS?

We just upgraded to 8.5.3 FP1 from 8.5.2 FP2 and that is when the server started to crash with this stack.

This server is clustered and it appears that the cluster has the same crash stack [ Fatal Thread (Panic) ].

Please let me know if you find something. I am going to contact IBM. The stack looks really similar to the one listed here: LO66393: DOMINO 8.5.3 SERVER CRASHING CONTINUOUSLY ON NSERVER PROCESS WITH DBGETNOTENONSUMMARYLENGTH FUNCTION CALLS but its not a perfect match and it appears to be specific to 8.5.3.