Domino 6.5 server crashing every couple minutes

HCL Domino Domino Forum
1

I am desperate for help! Can anyone help? I have recently upgraded a Domino server from 6.0.3 to 6.5, and it’s been crashing ever since. Here’s the fatal thread:

############################################################

FATAL THREAD 64/67 [ nSERVER:07bc: 1768]

FP=0x0731fb6c, PC=0x60194a3e, SP=0x0731fb60, stksize=12

EAX=0x4ae410ce, EBX=0x00000310, ECX=0x00004356, EDX=0x00000000

ESI=0x00000001, EDI=0x078e10b0, CS=0x0000001b, SS=0x00000023

DS=0x00000023, ES=0x00000023, FS=0x00000038, GS=0x00000000 Flags=0x00010293

Exception code: c0000005 (ACCESS_VIOLATION)

############################################################

@[ 1] 0x60194a3e nnotes._GetCounterOffset+30 (78e10b0,310,3ef82cc,28)

@[ 2] 0x601945d0 nnotes._Win32ProcessStats@4+320 (3ef82cc,3ef82cc,ffb9b0,0)

@[ 3] 0x601940d8 nnotes._PSGetDominoProcessInfo@4+1000 (3ef82cc,3ef8824,2ad31e8,3ef8c34)

@[ 4] 0x1005ccb5 nserverl._UpdatePerfList@4+501 (f01040ae,731fea8,1005b324,2ad31f0)

@[ 5] 0x1005c22e nserverl._LISTSRCUpdate@4+14 (2ad31f0,3ef82cc,731fec0,1005b2db)

@[ 6] 0x1005b324 nserverl.__PSUpdatePSHeadL@4+36 (3ef82cc,3ef0000,3ef82cc,3ef8c34)

@[ 7] 0x1005b2db nserverl._PSUpdatePSHead@4+43 (3ef82cc,59780001,3ef82cc,1)

@[ 8] 0x10057cda nserverl.__PSUpdatePerfStats@4+58 (3ef82cc,59780001,231838c,6d22003c)

@[ 9] 0x10057c1a nserverl._PSMain@4+90 (3ef82cc,0,3e8,fe6ebc)

@[10] 0x1005aeb6 nserverl._PlatformStatsTask@8+582 (6d22003c,5c740001,3,848)

@[11] 0x100016cb nserverl._Scheduler@4+763 (0,848,0,0)

@[12] 0x60114924 nnotes._ThreadWrapper@4+212 (0,0,848,0)

[13] 0x7c4e987c KERNEL32.SetThreadExecutionState+551

############################################################

PASS 2 : FATAL THREAD with STACK FRAMES 64/67 [ nSERVER:07bc: 1768]

FP=0731fb6c, PC=60194a3e, SP=0731fb60, stksize=12

Exception code: c0000005 (ACCESS_VIOLATION)

############################################################

---------- Top of the Stack ----------

   # 0731fb60  00000000 00000004 000000e6 0731fd4c  |............L.1.|

@[ 1] 0x60194a3e nnotes._GetCounterOffset+30 (78e10b0,310,3ef82cc,28)

2

Subject: Domino 6.5 server crashing every couple minutes

Platform Stats is crashing. I have no idea why, especially since you say it was working in 6.0.3. If you don’t need this feature then you can disable it with the notes.ini setting PLATFORM_STATISTICS_DISABLED=1

3

Subject: RE: Domino 6.5 server crashing every couple minutes

That’s exactly what it was! I fixed it a few hours ago after going through the NSD dumps, and finding what was in common. I then found a technote on it, disabled it, and all is well. I was just going to post what I found, when I saw your post. Thanks BTW!

I’m not sure that Platform Stats. was ever working. Here’s the full story. I tried the 6.0.3 incremental installer to go from 6.0.2cf1 to 6.0.3 It appeared to work, gave no errors, but then the server began crashing all the time! I’m guessing that it somehow enabled platform stats now that I look back at it. After being extremely frustrated, I tried everything, and one of the things was going to 6.5 This is a test server, so I figured what the heck. After that didn’t do it, I restored from backups, I tried a million different things, while going through the NSD dumps, when I finally figured it out. What a ride!

4

Subject: RE: Domino 6.5 server crashing every couple minutes

From de R6.5 Administrator Help:System configuration issue for platform statistics on Windows NT and Windows 2000 systems

On Windows NT and Windows 2000, an error may occur when loading certain performance dlls. If they do not function properly or take too long to pass data, the operating system automatically adds a value to the following Performance registry subkey where TypeOfPerfService may be PerfProc, PerfOS, or NoteStat:

HKEY_LOCAL_MACHINE\SYSTEM\Services\CurrentControlSet<TypeOfPerfService>\Performance.

When the error occurs, the value for the variable “Disable Performance Counters” is set to 1, which disables performance counters for statistics such as CPU utilization (Platform.System.PctCombinedCpuUtil) or Memory (Platform.Memory.PagesPerSec). These counters are found under the services PerfOS, PerfDisk, PerfProc and PerfNet.

If these statistic counters cannot be located, you may get the following error message, printed to both the event log and the console:

Platform Stats: _PSHandleDefaultCmd() Unable to set up default counters error =…"

Although the system may have set the “Disable Performance Counters” variable under a period of extreme stress on the system, once it has been set, this variable continues to disable all performance counters relating to its.dll, until it is manually set back to zero or deleted.

To reset the default counters, search the registry for the phrase “Disable Performance Counters.” If it occurs under PerfOS, PerfDisk, PerfProc or PerfNet, manually set it back to zero or delete the entire variable.

In my case, yesterday I had to use Platform_Statistics_Disabled=1 to workaround the problem. But today, after deleting from de register the key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControl\SetServices\PerfProc\Performance\DisablePerformanceCounter and the varible Platform_Statistics_Disabled=1 from notes.ini, everything works perfectly.

Hope this help anybody.

Regards