R6.5.4 FP2 Server Crashing every 2 hours - pls help

Can anyone help? I have a Domino mail server running R6.5.4 FP 2 on Windows 2k. Ever since I upgraded to R6.5.4 from R6.0.3 it has crashed every couple of weeks, but now this has suddenly got worse and the server won’t stay up for more than a couple of hours.

Suspecting Http or Ldap I have disabled these tasks and the server still crashes. I have looked at my NSD logs, and can not obviously see any reason for this problem. I also have verbose logging turned on, and this is not helping either.

I did see some maybe unrelated errors the first couple of crashes to “semaphore not allocated or invalid”, but I have not seen this error since and the server still crashes.

Any help would very much be appreciated?

Here’s the extract from my NSD Log:


INFO (0): Using PSAPI DLL

search path: .;C:\WINNT

Host Name : CFBTSR3

User Name : SYSTEM

Date : Tue Jan 10 00:39:35 2006

Windows Dir : C:\WINNT

Arguments : “C:\Lotus\Domino\nsd.exe” -dumpandkill -termstatus 5

OSVersion : Windows 2000 5.0 (Build 2195), PlatID=2, Service Pack 4 (2 Processors)

INFO (0): Found 20 Notes processes, matched 19

INFO (0): failed to match all of the Notes processes

→ Instance# 1 : (current)

            Ini File   : c:\lotus\domino\notes.ini

            Data Dir   : d:\lotus\domino\data

            Prog Dir   : C:\Lotus\Domino

            Notes Build: Release 6.5.4FP2 September 12, 2005      

        Proc List:

              C:\Program Files\CA\SharedComponents\BrightStor\DBAcommon\DBASVR.exe (2d0)

              C:\Lotus\Domino\nSERVER.EXE (750)

              C:\Lotus\Domino\nevent.EXE (40c)

              C:\Lotus\Domino\nReplica.EXE (7f8)

              C:\Lotus\Domino\nRouter.EXE (830)

              C:\Lotus\Domino\nUpdate.EXE (8b0)

              C:\Lotus\Domino\nAMgr.EXE (8d0)

              C:\Lotus\Domino\nAdminp.EXE (8e4)

              C:\Lotus\Domino\nSched.EXE (8ec)

              C:\Lotus\Domino\nCalConn.EXE (8f8)

              C:\Lotus\Domino\nHTTP.EXE (904)

              C:\Lotus\Domino\nMAPS.EXE (90c)

              C:\Lotus\Domino\nLDAP.EXE (914)

              C:\Lotus\Domino\nSMTP.EXE (91c)

              C:\Lotus\Domino\nAMgr.EXE (9b0)

              C:\Lotus\Domino\nAMgr.EXE (9b8)

              C:\Lotus\Domino\nmtc.EXE (9cc)

              C:\Lotus\Domino\nCldbdir.EXE (a20)

              C:\Lotus\Domino\nClrepl.EXE (a38)

DBG(0b50) 00:39:41

@@@@@@@@@@@@@@@@@ Process Table @@@@@@@@@@@@@@@@@

Active Users:

UID Name

0  SYSTEM

500 Administrator

INFO PID PPID UID STIME COMMAND

    0000 0000     0         ??? []

    0008 0000     0         ???   [       ?:0008]

    00cc 0008     0 01/09 22:07:05     [    smss:00cc]

    00e4 00cc     0 01/09 22:07:09       [   csrss:00e4]

    00e0 00cc     0 01/09 22:07:10       [winlogon:00e0]

    0118 00e0     0 01/09 22:07:11         [services:0118]

    01d8 0118     0 01/09 22:07:13           [ svchost:01d8]

    01f8 0118     0 01/09 22:07:13           [ spoolsv:01f8]

    0214 0118     0 01/09 22:07:13           [   msdtc:0214]

    028c 0118     0 01/09 22:07:14           [awhost32:028c]

    02bc 0118     0 01/09 22:07:23           [lic98rmt:02bc]

 -> 02d0 0118     0 01/09 22:07:23           [  DBASVR:02d0]

    02f8 0118     0 01/09 22:07:24           [ dcevt32:02f8]

    030c 0118     0 01/09 22:07:24           [dcstor32:030c]

    0348 0118     0 01/09 22:07:30           [DKService:0348]

    06e0 0348     0 01/10 00:39:39             [DfrgNTFS:06e0]

    0354 0118     0 01/09 22:07:30           [ svchost:0354]

    0374 0118     0 01/09 22:07:30           [  llssrv:0374]

    03a0 0118     0 01/09 22:07:30           [LogWatNT:03a0]

    03bc 0118     0 01/09 22:07:30           [mr2kserv:03bc]

    03ec 0118     0 01/09 22:07:30           [  regsvc:03ec]

    0404 0118     0 01/09 22:07:30           [  MSTask:0404]

    0434 0118     0 01/09 22:07:38           [ omaws32:0434]

    0674 0434     0 01/09 22:07:53             [ diagorb:0674]

    0444 0118     0 01/09 22:07:38           [    snmp:0444]

    0474 0118     0 01/09 22:07:39           [   VxSvc:0474]

    04a4 0118     0 01/09 22:07:39           [ WinMgmt:04a4]

    04b0 0118     0 01/09 22:07:39           [ svchost:04b0]

    052c 0118     0 01/09 22:07:41           [  Dfssvc:052c]

    0694 0118     0 01/09 22:07:54           [ svchost:0694]

  * 0558 0118     0 01/09 22:18:30           [nservice:0558]

 -> 0750 0558     0 01/09 22:18:32             [ nSERVER:0750]

 -> 040c 0750     0 01/09 22:18:37               [  nevent:040c]

 -> 07f8 0750     0 01/09 22:18:43               [nReplica:07f8]

 -> 0830 0750     0 01/09 22:18:43               [ nRouter:0830]

 -> 09cc 0830     0 01/09 22:18:45                 [    nmtc:09cc]

 -> 08b0 0750     0 01/09 22:18:43               [ nUpdate:08b0]

    0488 08b0     0 01/10 00:39:34                 [     nsd:0488]

 -> 08d0 0750     0 01/09 22:18:43               [   nAMgr:08d0]

 -> 09b0 08d0     0 01/09 22:18:45                 [   nAMgr:09b0]

 -> 09b8 08d0     0 01/09 22:18:45                 [   nAMgr:09b8]

 -> 08e4 0750     0 01/09 22:18:43               [ nAdminp:08e4]

 -> 08ec 0750     0 01/09 22:18:43               [  nSched:08ec]

 -> 08f8 0750     0 01/09 22:18:43               [nCalConn:08f8]

 -> 0904 0750     0 01/09 22:18:43               [   nHTTP:0904]

 -> 090c 0750     0 01/09 22:18:43               [   nMAPS:090c]

 -> 0914 0750     0 01/09 22:18:43               [   nLDAP:0914]

 -> 091c 0750     0 01/09 22:18:43               [   nSMTP:091c]

 -> 0a20 0750     0 01/09 22:18:55               [nCldbdir:0a20]

 -> 0a38 0750     0 01/09 22:19:05               [ nClrepl:0a38]

    0124 00e0     0 01/09 22:07:11         [   lsass:0124]

    064c 03dc   500 01/09 22:08:37 [Explorer:064c]

    0730 064c   500 01/09 22:08:49   [     mmc:0730]

INFO (0): Starting Debugger

** Attach to [ DBASVR:02d0]

=== Attach to C:\Program Files\CA\SharedComponents\BrightStor\DBAcommon\DBASVR.exe (0x2d0)

Load module: base=0x00400000, size= 163840 C:\Program Files\CA\SharedComponents\BrightStor\DBAcommon\DBASVR.exe

Load module: base=0x77f80000, size= 503808 C:\WINNT\system32\ntdll.dll

Load module: base=0x25600000, size= 135168 C:\Program Files\CA\SharedComponents\BrightStor\DBAcommon\CSTool.dll

Load module: base=0x75050000, size= 32768 C:\WINNT\system32\WSOCK32.DLL

Load module: base=0x7c570000, size= 733184 C:\WINNT\system32\KERNEL32.dll

Load module: base=0x75030000, size= 81920 C:\WINNT\system32\WS2_32.DLL

Load module: base=0x78000000, size= 282624 C:\WINNT\system32\MSVCRT.dll

Load module: base=0x7c2d0000, size= 401408 C:\WINNT\system32\ADVAPI32.dll

Load module: base=0x77d30000, size= 450560 C:\WINNT\system32\RPCRT4.DLL

Load module: base=0x75020000, size= 32768 C:\WINNT\system32\WS2HELP.DLL

Load module: base=0x74ff0000, size= 73728 C:\WINNT\system32\MSWSOCK.DLL

Load module: base=0x77e10000, size= 389120 C:\WINNT\system32\USER32.dll

Load module: base=0x77f40000, size= 233472 C:\WINNT\system32\GDI32.dll

Load module: base=0x77980000, size= 147456 C:\WINNT\system32\DNSAPI.DLL

Load module: base=0x76620000, size= 69632 C:\WINNT\system32\MPR.DLL

Load module: base=0x73930000, size= 65536 C:\WINNT\system32\CLUSAPI.DLL

Load module: base=0x10000000, size= 278528 C:\Program Files\CA\BrightStor ARCserve Backup Agent for Lotus Domino\dbanotes.dll

Load module: base=0x23800000, size= 135168 C:\Program Files\CA\SharedComponents\BrightStor\DBAcommon\CHEYPROD.dll

Load module: base=0x20c00000, size= 458752 C:\Program Files\CA\SharedComponents\BrightStor\DBAcommon\ASBRDCST.dll

Load module: base=0x00a60000, size= 53248 C:\Program Files\CA\SharedComponents\BrightStor\DBAcommon\BRAND.dll

Load module: base=0x780c0000, size= 397312 C:\WINNT\system32\MSVCP60.dll

Load module: base=0x77a50000, size= 966656 C:\WINNT\system32\ole32.dll

Load module: base=0x779b0000, size= 634880 C:\WINNT\system32\OLEAUT32.dll

Load module: base=0x60000000, size= 13791232 C:\Lotus\Domino\nNOTES.dll

Load module: base=0x621b0000, size= 1400832 C:\Lotus\Domino\nxmlpar.dll

Load module: base=0x62320000, size= 98304 C:\Lotus\Domino\nxmlcommon.dll

Load module: base=0x62150000, size= 380928 C:\Lotus\Domino\js32.dll

Load module: base=0x62350000, size= 999424 C:\Lotus\Domino\NLSCCSTR.DLL

Load module: base=0x719b0000, size= 32768 C:\WINNT\system32\SHFOLDER.dll

Load module: base=0x76b30000, size= 253952 C:\WINNT\system32\comdlg32.dll

Load module: base=0x70a70000, size= 413696 C:\WINNT\system32\SHLWAPI.DLL

Load module: base=0x71710000, size= 540672 C:\WINNT\system32\COMCTL32.dll

Load module: base=0x782f0000, size= 2392064 C:\WINNT\system32\SHELL32.dll

Load module: base=0x62950000, size= 24576 C:\Lotus\Domino\ndgts.dll

Load module: base=0x77820000, size= 28672 C:\WINNT\system32\VERSION.dll

Load module: base=0x759b0000, size= 24576 C:\WINNT\system32\LZ32.DLL

Load module: base=0x75170000, size= 323584 C:\WINNT\system32\NETAPI32.dll

Load module: base=0x7c340000, size= 61440 C:\WINNT\system32\Secur32.dll

Load module: base=0x751c0000, size= 24576 C:\WINNT\system32\NETRAP.DLL

Load module: base=0x75150000, size= 61440 C:\WINNT\system32\SAMLIB.DLL

Load module: base=0x77950000, size= 172032 C:\WINNT\system32\WLDAP32.DLL

Load module: base=0x00ec0000, size= 49152 C:\Program Files\CA\BrightStor ARCserve Backup Agent for Lotus Domino\notesres.dll

Load module: base=0x624d0000, size= 892928 C:\Lotus\Domino\NSTRINGS.DLL

Load module: base=0x625b0000, size= 24576 C:\Lotus\Domino\namhook.DLL

Load module: base=0x1f140000, size= 77824 C:\Lotus\Domino\DBAPHook.dll

Load module: base=0x1f270000, size= 352256 C:\CA_LIC\lic98.dll

############################################################

thread 1/7: [ DBASVR:02d0:02cc]

FP=0012fc80, PC=77f8c55d, SP=0012fc10, stkbase=00030000, stksize=8192

############################################################

[ 1] 0x77f8c55d ntdll.ZwReadFile+11 (7c,12fd48,216,12fca8)

[ 2] 0x7c2e0135 ADVAPI32.StartServiceCtrlDispatcherW+1289 (7c,12fd48,216,12fce4)

[ 3] 0x7c2dffbb ADVAPI32.StartServiceCtrlDispatcherW+911 (7c,12fd48,216,6f944)

[ 4] 0x7c2e1995 ADVAPI32.StartServiceCtrlDispatcherA+114 (12ff70,41f560,40c35b,0)

[ 5] 0x0040c346 DBASVR+49990 (1,640c10,640c68,6f944)

[ 6] 0x0040f1bd DBASVR+61885 (6f944,77fcc35c,7ffdf000,21a)

[ 7] 0x7c5987e7 KERNEL32.FindFirstChangeNotificationA+15941 (40f109,0,c8,100)

############################################################

thread 2/7: [ DBASVR:02d0:02ec]

FP=0074fda0, PC=77f94091, SP=0074fd7c, stkbase=00650000, stksize=4096

############################################################

[ 1] 0x77f94091 ntdll.NtWaitForSingleObject+11 (bc,ffffffff,0,40ca89)

[ 2] 0x7c57b3d5 KERNEL32.CreateProcessW+387 (1,133490,12f9a4,7c57b382)

[ 3] 0x7c2e02f7 ADVAPI32.StartServiceCtrlDispatcherW+1739 (0,0,0,0)

############################################################

thread 3/7: [ DBASVR:02d0:02f0]

FP=0084ff1c, PC=77f94091, SP=0084fef8, stkbase=00750000, stksize=16384

############################################################

[ 1] 0x77f94091 ntdll.NtWaitForSingleObject+11 (88,ffffffff,0,77d523f9)

[ 2] 0x7c57b3d5 KERNEL32.CreateProcessW+387 (5,7c2d6c82,0,0)

[ 3] 0x77d38ba0 RPCRT4.NDRCContextMarshall+6968 (1,139218,0,40bc96)

[ 4] 0x77d3aeb5 RPCRT4.NdrConformantStringBufferSize+89 (0,5,7c2d6c82,0)

[ 5] 0x7c57b382 KERNEL32.CreateProcessW+304 (40bba0,0,0,1)

############################################################

thread 4/7: [ DBASVR:02d0:02fc]

FP=0095fee4, PC=77f8beb2, SP=0095febc, stkbase=00860000, stksize=36864

############################################################

[ 1] 0x77f8beb2 ntdll.ZwRemoveIoCompletion+11 (8c,95ff1c,95ff0c,95ff14)

[ 2] 0x77d809da RPCRT4.NdrRpcSmClientFree+438 (493e0,95ff60,95ff5c,95ff70)

[ 3] 0x77d50ede RPCRT4.TowerConstruct+1336 (77d39a00,1394e0,84d27e,77f8c277)

[ 4] 0x77d50d17 RPCRT4.TowerConstruct+881 (13bc40,95ffec,7c57b382,13bf78)

[ 5] 0x77d41c6c RPCRT4.RpcBindingInqAuthClientExW+372 (13bf78,84d27e,77f8c277,13bf78)

[ 6] 0x7c57b382 KERNEL32.CreateProcessW+304 (0,0,0,0)

############################################################

thread 5/7: [ DBASVR:02d0:0b14]

FP=00fdfc5c, PC=77f9323e, SP=00fdfc10, stkbase=00ee0000, stksize=20480

############################################################

[ 1] 0x77f9323e ntdll.ZwWaitForMultipleObjects+11 (fdfc34,1,0,0)

[ 2] 0x7c599f6c KERNEL32.SetLocalTime+40 (0,0,0,0)

############################################################

thread 6/7: [ DBASVR:02d0:0ad0]

FP=1f4dffb4, PC=77f9323e, SP=1f4dff8c, stkbase=1f3e0000, stksize=4096

############################################################

[ 1] 0x77f9323e ntdll.ZwWaitForMultipleObjects+11 (0,95dc34,77f83404,0)

[ 2] 0x7c57b382 KERNEL32.CreateProcessW+304 (0,0,0,0)

############################################################

thread 7/7: [ DBASVR:02d0:0b98]

FP=1f51ffb4, PC=77fa144c, SP=1f51ffa8, stkbase=1f4e0000, stksize=8192

############################################################

[ 1] 0x77fa144c ntdll.DbgBreakPoint+1 (0,118fcbc,77f98191,0)

[ 2] 0x7c57b382 KERNEL32.CreateProcessW+304 (0,0,0,0)

** Attach to [ nSERVER:0750]

Subject: R6.5.4 FP2 Server Crashing every 2 hours - pls help

Hi,

You already created a thread with this subject - why create a second one??

What about the answers for the questions which were answered in the other thread??

As well is this the full NSD?? Need to post the area where you see the fatal stack.

Kjeld