Hi,
We have an Domino 6.5.6 Domino server on windows 2000 sever. There is no any user load other than domino task which are running in the Background. From last week, almost everyday it is getting crashed due to ‘PANIC: Cannot attach to shared memory region, due to insufficient access (probably owned by another user or group)’
Pls give some clues to anlyze which program is trying to attached domino;s shared memory?
ConstrainedSHM parmater is useful for this issue or not?
Pls help…
Here is some data from NSD. If you need any other data from NSD, pls let me know.
############################################################
thread 21/61: [ nhttp: 4032: 664] FATAL THREAD
FP=0d34fdc0, PC=60193297, SP=0d34f248, stkbase=0d250000, stksize=262144
############################################################
Got exception event: code=0xc0000005 (ACCESS_VIOLATION) from [ nhttp: 4032: 664]
flags = 0x0
Address = 0x60193297
Chain = 0x0
1st chance = NO
Access violation exception on write access to addr 0xffffffff
---------- Top of the Stack ----------
# 0d34f248 00000001 0d3401b1 00000bb8 54200a0a |......4....... T|
# 0d34f258 61657268 305b3d64 3a304346 30313030 |hread=[0FC0:0010|
# 0d34f268 3932302d 530a5d38 6b636174 73616220 |-0298].Stack bas|
# 0d34f278 78303d65 35334430 34393030 7453202c |e=0x0D350094, St|
# 0d34f288 206b6361 657a6973 33203d20 20323736 |ack size = 3672 |
# 0d34f298 65747962 41500a73 3a43494e 6e614320 |bytes.PANIC: Can|
# 0d34f2a8 20746f6e 61747461 74206863 6873206f |not attach to sh|
# 0d34f2b8 64657261 6d656d20 2079726f 69676572 |ared memory regi|
# 0d34f2c8 202c6e6f 20657564 69206f74 6675736e |on, due to insuf|
# 0d34f2d8 69636966 20746e65 65636361 28207373 |ficient access (|
# 0d34f2e8 626f7270 796c6261 6e776f20 62206465 |probably owned b|
# 0d34f2f8 6e612079 6568746f 73752072 6f207265 |y another user o|
# 0d34f308 72672072 2970756f 6000000a 01150014 |r group)...`....|
# 0d34f318 00000000 00000001 0d34f38c 600f6d66 |..........4.fm.`|
# 0d34f328 0d34f378 00000000 02851096 02851014 |x.4.............|
# 0d34f338 00000000 7a6e0014 00000000 00000000 |......nz........|
# 0d34f348 0d34fc9c 0065344b 0d34fca8 00414bc8 |..4.K4e...4..KA.|
# 0d34f358 7a6e0014 00481ffb 00000004 0b45f6d4 |..nz..H.......E.|
# 0d34f368 0b45f6b4 0b460024 0b460024 0000000d |..E.$.F.$.F.....|
# 0d34f378 0d34f38c 10027e94 10039e8c 0b45ff60 |..4..~......`.E.|
# 0d34f388 0000000d 0d34f3a8 10013663 10039e8c |......4.c6......|
# 0d34f398 0b45ff60 0000000d 10039e8c 0b46016c |`.E.........l.F.|
# 0d34f3a8 0d34f3c4 1001c710 10039e8c 1001c71b |..4.............|
# 0d34f3b8 0d34f43c 0b45f54c 015e8f30 0d34f3d0 |<.4.L.E.0.^...4.|
# 0d34f3c8 1001c75c 10039e8c 0d34f3dc 1001c74c |\.........4.L...|
# 0d34f3d8 10039e8c 0d34f3f0 1000c975 0b45f54c |......4.u...L.E.|
# 0d34f3e8 0d34f430 015e8f30 0d34f40c 1000e1f3 |0.4.0.^...4.....|
# 0d34f3f8 0d34f44c 0d34f414 015ba2b8 0d34f41c |L.4...4...[...4.|
# 0d34f408 0d34fbe2 6261f0ae 000000f5 0b45f708 |..4...ab......E.|
# 0d34f418 0b0b7610 0d34f458 0b0b72cb 0b45fdb8 |.v..X.4..r....E.|
# 0d34f428 0b0cbefc 0b45f6ec 00000002 0b45fdb8 |......E.......E.|
# 0d34f438 0b45fdd4 0b460260 0b46026c 00000000 |..E.`.F.l.F.....|
# 0d34f448 00000000 00000000 00000000 00000000 |................|
# 0d34f458 0d34f464 0b0b763d 0b45f708 0d34fca8 |d.4.=v....E...4.|
# 0d34f468 1000cef4 0d34fca8 1000d084 0d34f488 |......4.......4.|
# 0d34f478 00000800 00000000 00000003 0b45f6c4 |..............E.|
# 0d34f488 4c2f3a44 7375746f 6d6f442f 2f6f6e69 |D:/Lotus/Domino/|
# 0d34f498 61746144 7270612f 62657773 66736e2e |Data/aprsweb.nsf|
# 0d34f4a8 7a6d3000 0d34f4bc 0049cac7 7a6d3014 |.0mz..4...I..0mz|
# 0d34f4b8 7a6d3014 0d34f570 00505c1a 00000001 |.0mzp.4..\P.....|
# 0d34f4c8 00000000 6d23146c 0d34f47a 600014e3 |....l.#mz.4....`|
# 0d34f4d8 6b6210fa 6b6210f8 0d34f50c 6000313c |..bk..bk..4.<1.`|
# 0d34f4e8 6b6210f8 7c59c354 0d34f5a0 00000000 |..bkT.Y|..4.....|
# 0d34f4f8 f02408d5 00002305 00000e64 1001852a |..$..#..d...*...|
# 0d34f508 0b46038c 00000000 00000024 0d34fa9d |..F.....$.....4.|
# 0d34f518 6261f0af 00000025 0d34f534 623e0a3d |..ab%...4.4.=.>b|
# 0d34f528 0d34f568 00000000 0281c018 0d34f548 |h.4.........H.4.|
# 0d34f538 60007a64 0d34f568 00000025 0281c018 |dz.`h.4.%.......|
# 0d34f548 0d34f560 60007a37 0d34f568 00000025 |`.4.7z.`h.4.%...|
# 0d34f558 0281c018 000000f2 0d34f570 0d34fa9d |........p.4...4.|
# 0d34f568 6261f0af 0d34f6c8 0d34fa9d 0d34f6c8 |..ab..4...4...4.|
# 0d34f578 600112c1 0d34fa90 000000b8 000000d2 |...`..4.........|
# 0d34f588 0d34f904 00000000 00000000 00000000 |..4.............|
# 0d34f598 00000000 00000000 00000000 00000000 |................|
# 0d34f5a8 00000000 0066ab58 0d34f5c9 00000000 |....X.f...4.....|
# 0d34f5b8 00000000 00010000 00000000 00000000 |................|
# 0d34f5c8 00000000 00000000 00000000 00000000 |................|
# 0d34f5d8 00000000 00000000 00000000 00000000 |................|
# 0d34f5e8 00000000 00000000 00000000 00000000 |................|
# 0d34f5f8 00000000 00000000 00000000 00000000 |................|
# 0d34f608 00000000 00000000 0d34f89c 7800fd15 |..........4....x|
# 0d34f618 0000000a 0d34f8b4 0d34f888 00000800 |......4...4.....|
# 0d34f628 fffffffe 00000000 7800fc62 10039d7c |........b..x|...|
# 0d34f638 00000002 0d34f8b4 0d34f888 780238ae |......4...4..8.x|
# 0d34f648 7b5a01a8 015a157d 00000000 fffffffd |..Z{}.Z.........|
# 0d34f658 00000040 0d34f8c8 7800f6a8 00000030 |@.....4....x0...|
# 0d34f668 0d34f8e0 0d34f8b4 780238ae 7b5a01a8 |..4...4..8.x..Z{|
# 0d34f678 015a157d 00000000 74fd3bac 00000aac |}.Z......;.t....|
# 0d34f688 00000830 6261f0a4 0d34f714 00000000 |0.....ab..4.....|
# 0d34f698 00012024 0d34f6f8 00000000 00000000 |$ ....4.........|
# 0d34f6a8 00000000 00000000 00000000 00000000 |................|
# 0d34f6b8 00000000 0d34fa90 00000000 00000000 |......4.........|
# 0d34f6c8 0d34f6ec 6001114e 00000000 00000003 |..4.N..`........|
# 0d34f6d8 6261f0a4 00000000 00000000 00000000 |..ab............|
# 0d34f6e8 0d34f724 0d34f704 60014f39 0d34fa90 |$.4...4.9O.`..4.|
# 0d34f6f8 6261f0a4 0000ffff 0d34f714 0d34f734 |..ab......4.4.4.|
# 0d34f708 626182c1 0d34fa90 6261f0a4 000000b8 |..ab..4...ab....|
# 0d34f718 0d34f734 626182cd 0d34fa90 6000110d |4.4...ab..4....`|
# 0d34f728 6000114a 7a880086 00000fc0 7a880084 |J..`...z.......z|
# 0d34f738 0d34f748 600010be 60001216 6000127b |H.4....`...`{..`|
# 0d34f748 7a880086 7a880086 0d34f760 600014e3 |...z...z`.4....`|
# 0d34f758 7a880086 7a880084 0d34f794 60001b92 |...z...z..4....`|
# 0d34f768 7a880084 00720009 00000004 00000008 |...z..r.........|
# 0d34f778 01590086 000002b0 600014e3 0000000c |..Y........`....|
# 0d34f788 00000078 0d34fccc 00000000 0d34f7a8 |x.....4.......4.|
# 0d34f798 600019c4 00000514 000001a0 0d34f828 |...`........(.4.|
# 0d34f7a8 0d34f858 60001e26 60c54328 000001a0 |X.4.&..`(C.`....|
# 0d34f7b8 0d34f828 000001a0 015edb0a 00003a45 |(.4.......^.E:..|
# 0d34f7c8 00000460 000001c0 0d34f854 0d34f884 |`.......T.4...4.|
# 0d34f7d8 60001e26 60c5431c 000001c0 0d34f854 |&..`.C.`....T.4.|
# 0d34f7e8 000001c0 015eba8e 00003a45 0d34f950 |......^.E:..P.4.|
# 0d34f7f8 0d34fbe5 0d34f950 600112c1 0d34fbd8 |..4.P.4....`..4.|
# 0d34f808 000000b8 000000d2 0d34fb8c 00000000 |..........4.....|
# 0d34f818 00000000 0b45f9ac 00000000 00000036 |......E.....6...|
# 0d34f828 00000aa4 00000000 00000000 00000000 |................|
# 0d34f838 60c54318 00720009 ffffffff 00000000 |.C.`..r.........|
# 0d34f848 00080000 3000110d 00000005 6c3d4e43 |.......0....CN=l|
# 0d34f858 7373746f 6e31306f 63676374 2f656766 |otsso01ntcgcfge/|
# 0d34f868 65673d4f 656e6f6d 00000079 00000000 |O=gemoney.......|
# 0d34f878 00000000 00000000 00000000 015edb0a |..............^.|
# 0d34f888 00000173 00000002 10039d7c ffffffff |s.......|.......|
# 0d34f898 00000000 74fd1275 00000830 00000001 |....u..t0.......|
# 0d34f8a8 0d34f8c0 0d34fb50 0d34f948 0d34f938 |..4.P.4.H.4.8.4.|
# 0d34f8b8 7ffffe8b 7a880ab0 ffb3b4c0 ffffffff |.......z........|
# 0d34f8c8 0007ed50 00000000 00000000 0d34f9c0 |P.............4.|
# 0d34f8d8 74fd3c59 00000830 74fda186 00000abc |Y<.t0......t....|
# 0d34f8e8 0d34fb50 0d34fc54 01107e88 0d34fa4c |P.4.T.4..~..L.4.|
# 0d34f8f8 fff0bdc0 ffffffff 00000000 00000000 |................|
# 0d34f908 015a14fe 015a153b 10039d7c 00000000 |..Z.;.Z.|.......|
# 0d34f918 7b5a01a8 20343033 00000030 10050b44 |..Z{304 0...D...|
# 0d34f928 000000f4 0d34fbbc fff0bdc0 ffffffff |......4.........|
# 0d34f938 fff0bdc0 ffffffff 00000001 015eed00 |..............^.|
# 0d34f948 00000abc 00000008 00000000 6e6e6143 |............Cann|
# 0d34f958 6120746f 63617474 6f742068 61687320 |ot attach to sha|
# 0d34f968 20646572 6f6d656d 72207972 6f696765 |red memory regio|
# 0d34f978 64202c6e 74206575 6e69206f 66667573 |n, due to insuff|
# 0d34f988 65696369 6120746e 73656363 70282073 |icient access (p|
# 0d34f998 61626f72 20796c62 656e776f 79622064 |robably owned by|
# 0d34f9a8 6f6e6120 72656874 65737520 726f2072 | another user or|
# 0d34f9b8 6f726720 00297075 0006ef98 77fcb7c2 | group)........w|
# 0d34f9c8 00071378 000721d0 00000000 77fcb6ca |x....!.........w|
# 0d34f9d8 00000000 00000000 00000000 00000000 |................|
# 0d34f9e8 00000000 00000000 00000000 00390038 |............8.9.|
# 0d34f9f8 003b0000 003d003c 003f003e 000721d0 |..;.<.=.>.?..!..|
# 0d34fa08 00430042 00450044 00470046 00000000 |B.C.D.E.F.G.....|
# 0d34fa18 004b004a 004d004c 0006ed20 00510000 |J.K.L.M. .....Q.|
# 0d34fa28 77fcd3dd ffffffff 01140000 01140000 |...w............|
# 0d34fa38 01144fe0 00000201 77fcb0ff 0d34fbcc |.O.........w..4.|
======================================
---- Notes Memory Analyzer (memcheck) → MM/OS Structure Information (Time 01:01:47) ------ @@>
Start Time = 01/31/2008 06:21:37 PM
Crash Time = 02/02/2008 01:01:38 AM
Error Message = PANIC: Cannot attach to shared memory region, due to insufficient access (probably owned by another user or group)
SharedDPoolSize = 4194304
FaultRecovery = 0x00010010
Cleanup Script Timeout= 300
Crash Limits = 3 crashes in 5 minutes
StaticHang = [ nhttp: 4032: 16]/[ nhttp: 4032: 664] (0xfc0/0x10/0x298)
ConfigFileSem = ( SEM:#0:0x010d) n=0, wcnt=-1, Users=-1, Owner=[ : 0]
FDSem = ( RWSEM:#11:0x410f) rdcnt=-1, refcnt=0 Writer=[ : 0], n=11, wcnt=-1, Users=0, Owner=[ : 0]
=====================================
<@@ ------ Notes Memory Analyzer (memcheck) → Shared Memory Stats (Time 18:30:14) ------ @@>
TYPE : Count SIZE ALLOC FREE FRAG OVERHEAD %used %free
Static-DPOOL: 28 85983232 79017524 6931904 0 53440 91% 8%
Overall : 28 85983232 79017524 6931904 0 53440 91% 8%
<@@ ------ Notes Memory Analyzer (memcheck) → Pool Analysis (Time 18:30:14) ------ @@>
<@@ ------ Notes Memory Analyzer (memcheck) → Pool Analysis → Shared Handle Table Info (Time 18:30:14) ------ @@>
Handles = 624
MaxHandle = 263815
Chunks = 39
FirstFreeHandle = 584
MaxUsedHandle = 610
MaxUsedMemHandle = 263815
MemHandles = 1680
FirstFreeMemHandle = 263795
MemChunks = 105
FirstMemHandle = 262144
FirstMemChunk = 16384
FreeQueue = ( RWSEM:#1:0x4117) rdcnt=-1, refcnt=0 Writer=[ : 0], n=1, wcnt=-1, Users=0, Owner=[ : 0]
Num Handles= 624, Max Handles= 262143, Avail=261519
Num MemHandles= 1680, Max MemHandles= 1048576, Avail=1046896
======================================
<@@ ------ Notes Memory Analyzer (memcheck) → Memory Usage Summary → Top 10 Shared Memory Block Usage (Time 18:30:14) ------ @@>
BY SIZE
Type TotalSize Handles Typename
0x82cd 59043840 15 BLK_UBMBUFFER
0x834a 3145728 3 BLK_GB_CACHE
0x826c 1635150 25 BLK_EXTMGR
0x8f56 1177308 18 BLK_ISERV_CONFIG_RECORDS
0x8f57 1177308 18 BLK_ISERV_CONFIG_PARAMS
0x82cc 1094400 15 BLK_UBMBCB
0x8252 1048576 1 BLK_NSF_POOL
0x826d 1048576 1 BLK_NSF_DIRMANPOOL
0x8433 1048576 1 BLK_LDAP_POOL
0x8311 1048576 1 BLK_NIF_POOL
BY HANDLE COUNT
Type Handles TotalSize Typename
0x841c 360 917268 BLK_VARRAY_CHUNK
0x841b 355 14244 BLK_VARRAY
0x8439 245 267922 BLK_BPOOL_PERPROCESS_INFO
0x8604 176 1966 BLK_TEXTLIST
0x8001 115 155710 ???
0x8405 102 159594 BLK_IDTABLE
0x82d2 86 106156 BLK_BUCKETDESCPAGE
0x8219 86 549222 BLK_UNK
0x815f 69 14238 BLK_FILE_DESCRIPTOR_GROUP
0x8234 43 176128 BLK_BITMAP
<@@ ------ Notes Memory Analyzer (memcheck) → MM/OS Structure Information (Time 18:30:14) ------ @@>
Start Time = 01/31/2008 06:21:37 PM
Crash Time = 01/31/2008 06:29:44 PM
SharedDPoolSize = 4194304
FaultRecovery = 0x00010010
Cleanup Script Timeout= 300
Crash Limits = 3 crashes in 5 minutes
ConfigFileSem = ( SEM:#0:0x010d) n=0, wcnt=-1, Users=-1, Owner=[ : 0]
FDSem = ( RWSEM:#11:0x410f) rdcnt=-1, refcnt=0 Writer=[ : 0], n=11, wcnt=-1, Users=0, Owner=[ : 0]
<@@ ------ Notes Memory Analyzer (memcheck) → SEM Info (Time 18:30:14) ------ @@>
SpinLockIterations = 1500
FirstFreeSem = 344
SemTableSize = 347
<@@ ------ Notes Memory Analyzer (memcheck) → Process Table (Time 18:30:14) ------ @@>
PID=[ nhttp: 4032], NestCnt=1
PID=[ nserver: 3628], NestCnt=1
PID=[ nevent: 4600], NestCnt=1
PID=[nreplica: 3432], NestCnt=1
PID=[ nadminp: 3180], NestCnt=1
PID=[ nupdate: 372], NestCnt=1
PID=[ nsched: 3644], NestCnt=1
PID=[ncalconn: 1268], NestCnt=1
PID=[ nrouter: 3868], NestCnt=1
PID=[ namgr: 3004], NestCnt=1
PID=[ nimap: 3500], NestCnt=1
PID=[ nldap: 3204], NestCnt=1
PID=[ npop3: 2176], NestCnt=1
PID=[ nsmtp: 3888], NestCnt=1
PID=[nddmtran: 3952], NestCnt=1
PID=[ namgr: 4408], NestCnt=2
PID=[ namgr: 1312], NestCnt=2
PID=[ namgr: 3932], NestCnt=2
PID=[ namgr: 3360], NestCnt=2
PID=[ namgr: 2996], NestCnt=2
PID=[ namgr: 552], NestCnt=2
PID=[ namgr: 4080], NestCnt=2
PID=[ namgr: 448], NestCnt=2
PID=[ namgr: 3636], NestCnt=2
PID=[ namgr: 2840], NestCnt=2