Domino crash for a process using high memory and the limit of 2GB memory for Win32

hcl-bot · November 13, 2006, 3:54am

HiOne of application on domino crashed. The analysis from IBM is the process used 2GB memory and this indicates a problem criticizes because of the limit of 2GB of Windows 32. The lack of memory caused for high allocation of particular memory of the nSMDreal caused the PANIC: Cannot attach you shared memory region, due you insufficient access (probably owned by to another to user or group).

From NSD, nsmdreal used about 336MB private memory and 1.6GB shared memroy.

My question are,

Why this crash occured, is it really because the 2GB limintation of Win32?
As I know, a process cannot allocate more than 2GB memory from OS. When it ties to allocate more than 2GB memory from OS, OS will return a NULL pointer and it indicates failed. The process should handle this error and should not crash. From the NSD, it seems that the reason of the crash isn’t the process using a NULL pointer. It seems the problem existed Domino managing shared memory. If my analysis is wrong, please correct me or could you explain detailedly how domino manage memory at the current situation.

Thanks!

Here are part of NSD

############################################################

thread 6/9: [nsmdreal: 2508: 1a54]

FP=4f533db0, PC=7c82ed54, SP=4f533d40

stkbase=4f540000, total stksize=397312, used stksize=49856

############################################################

@[ 1] 0x7c82ed54 ntdll.KiFastSystemCallRet+0 (4964,493e0,0,4f534230)

@[ 2] 0x77e6ba42 KERNEL32.WaitForSingleObjectEx+34 (4964,493e0,0,0)

@[ 3] 0x601a7c9e nnotes._OSRunExternalScript@4+558 (4f534348)

@[ 4] 0x601a69e6 nnotes._OSFaultCleanup@12+422 (0,0,0)

@[ 5] 0x601b15c7 nnotes._OSNTUnhandledExceptionFilter@4+183 (4f53506c)

@[ 6] 0x4af10e78 nSctHook.__CxxUnhandledExceptionFilter+62 (4f53506c,77e6b831,4f535074,0)

@[ 7] 0x77e84259 KERNEL32.LocalFlags+1912 (4b12fe80,12ef70,0,fffe0129)

############################################################

FATAL THREAD 6/9 [nsmdreal: 2508: 1a54]

FP=0x4f535fb8, PC=0x601933c7, SP=0x4f535440

stkbase=4f540000, total stksize=397312, used stksize=43968

EAX=0x010f088c, EBX=0x00000000, ECX=0x008e0000, EDX=0x008e0000

ESI=0x4f535b4c, EDI=0x00000000, CS=0x0000001b, SS=0x00000023

DS=0x00000023, ES=0x00000023, FS=0x0000003b, GS=0x00000000 Flags=0x00010202

Exception code: c0000005 (ACCESS_VIOLATION)

############################################################

@[ 1] 0x601933c7 nnotes._Panic@4+631 (60a30013)

@[ 2] 0x601930fc nnotes._Halt@4+28 (4f5301b1)

@[ 3] 0x601159ad nnotes._AccessAllProtected@0+77 ()

@[ 4] 0x6004ff1f nnotes._AccessAll@8+47 (1,1)

@[ 5] 0x60050e15 nnotes._ProcessGlobalEvent@4+21 (d22cfc)

@[ 6] 0x60050d21 nnotes._OSProcessShouldQuit@0+49 ()

Invalid stack frame detected: Invalid frame pointer (BP): 12ef70

<@@ Performance Data → Process Memory Mappings: [nsmdreal: 2508] (summary) (Time 13:55:20) @@>

COMMIT RESERVED TOTAL

PRIVATE: 196.4M 139.6M 336.1M

MAPPED: 1.6G 4.6M 1.6G

IMAGE: 52.0M 0.0K 52.0M

TOTAL: 1.8G 144.3M 1.9G

Totals: Used= 1.9G, Free=57.0M, Fragmented=143.3M, Overall= 2.0G, maxFree= 0.0K, Limit=7ffeffff ( 2.0G)

hcl-bot · November 13, 2006, 4:51am

Subject: Domino crash for a process using high memory and the limit of 2GB memory for Win32

A possible workaround in Domino to alleviate the crashes is to reduce shared memory allocation by Domino processes. This will allow greater private process memory allocation by the third party task prior to the 2GB limit being reached and Domino crashing.

Example parameters to control shared memory allocation in Domino are:

NSF_BUFFER_POOL_SIZE_MB=

ConstrainedSHMSizeMB=

hcl-bot · November 13, 2006, 5:13am

Subject: RE: Domino crash for a process using high memory and the limit of 2GB memory for Win32

Yes, these parameters can alleviate the crashes. But it seems to impact the whole domino performance. After all, the shared memory on this server is 1.6GB. It’s very high. It means the working load on this server is also very high. If adding these two paramaters to control shared memory, does it impact the server’s performance? I cannot find the official doc about the relationship of these paramaters setting and domino server performance. Another, why this 1.6GB shared memory is mapped to all processes on domino server, even if the process doesn’t use so many shared memory at all. That means when a precess starting it at least uses 1.6GB memory on this domino server.

hcl-bot · November 13, 2006, 10:02am

Subject: Domino crash for a process using high memory and the limit of 2GB memory for Win32

It is not usual for a process to use such an amount of memory.The problem should be in nSDMReal.exe.

I suggest contacting them with the .nsd

hcl-bot · November 13, 2006, 11:02am

Subject: Domino crash for a process using high memory and the limit of 2GB memory for Win32

Your best bet is to contact Trend Micro since they created the application that is causing your server to crash.

hcl-bot · November 13, 2006, 9:54pm

Subject: RE: Domino crash for a process using high memory and the limit of 2GB memory for Win32

We have contact TrendMicro and IBM to analyze this problem.The analyzed result by IBM is the crash is caused by smdreal.exe using high memory and the limit of 2GB memory for Win32. The suggested action is to add “NSF_BUFFER_POOL_SIZE_MB=512”. After adding this paramater, it seems the shared memory usage for domino server doesn’t exceed 1.5GB, But last time crash, the shared memory is 1.6GB. It seems the problem exists domino server manager shared memory, not the process itself. After all, an application cannot manage the shared memory on domino server.

hcl-bot · November 14, 2006, 12:33am

Subject: RE: Domino crash for a process using high memory and the limit of 2GB memory for Win32

There are a couple of other notes.ini parameters to adjust. The first step would be to use something like ConstrainedSHMSizeMB=1536 (1.5GB), or possibly 1280 (1.25GB). That sets a hard limit to the maximum amount of shared memory Domino can use. NSF_BUFFER_POOL_SIZE_MB is just the NSF buffer pool, there’s a lot more things that go into the shared memory space. Here’s presentation from Lotusphere 2006 that goes into detail: http://www-10.lotus.com/ldd/sandbox.nsf/ByDate/cb533a62695b1f08852571370053fa43?OpenDocument

hcl-bot · November 14, 2006, 3:45am

Subject: RE: Domino crash for a process using high memory and the limit of 2GB memory for Win32

Thanks for these information. It help understand domino managing memory. The paramater “ConstrainedSHMSizeMB” can control shared memory and I also know, in domino 7, it can pre-allocate the shared memory and never again create any more shared memory, thus preventing the possibility of a later process allocating shared memory that other processes cannot map into their address space. So using some control memory settings can avoid the crash. But my concerns for submitting this subject are 1. If this control memory setting will impact the domino performance. After all, last time crash, domino server allocated more than 1.6GB shared memory. I assume with the working load increase, the shared memory increase too.

2. Why this crash happen at this situation, from the callstack, it seems domino server let the process crash actively in this situation. It seems the problem exists domino managing memory.  Whatever, domino shuould report an error instead of crash.