Subject: RE: Memory leak - ND6/W2KSP3
Nor is it fixed in 6.5. I took the unprecedented step of installing this last night in hopes it would take care of this problem. It didn’t.
The only thing I’ve had any success with is minimizing the console on the server. perfmon reports that all the nblahblah processes give back memory as soon as you minimize the console - then they start taking back!
Subject: RE: Memory leak - ND6/W2KSP3
Looking at your description it could be that the properties of your server console needs to be changed.1. Properties from upper left hand corner of server console.
Disable QuickEdit Mode, and OK.
Save properties for future windows with the same title, and OK.
Subject: RE: Memory leak - ND6/W2KSP3
We are a single server 140 user shop. Here are some statistics from when we were running 6.0 under NT4 with 512 MB of RAM and McAfee GroupShield 5.21. 512 MB worked under Domino 5 until we upgraded to 6 and installed GroupShield 5.21, and the RAM usage skyrocketed. Here are statistics from that time:
4:24 PM … Shortly after server restart
total: 604MB (“Mem usage” at bottom of taskman)
nserver 69MB
nupdate 48MB
nrouter 42MB
8:24 AM … Next morning when server stopped responding
total: 646MB
nserver 176MB
nupdate 161MB
nrouter 89MB
Now after this point we did everyhing we could think of to solve the problem. Lotus support said we didn’t have anough RAM. So we upgraded to 1GB, formatted the OS partition and installed Windows 2000 Server SP4, upgraded to 6.02CF1, removed GroupShield Domino, stopped using the blackhole feature and outsourced all of our virus/spam scanning, and increased the swap size to 1.5GB. The mem usage numbers are different (higher, probably due to Win2k vs NT4) but we are still having the same problem.
Can anyone explain this one: Look at the numbers I gave above. The differences between the “total” memory used at the two times is smaller than the difference of nrouter, nserver, and nupdate combined. What gives? I don’t have exact numbers for our current config, but they’re something like 890 MB when it crashes, but if you add up nrouter, nserver, and nupdate they total 1200MB alone, without including any other tasks in taskman. Can anyone tell me what this means?
Subject: RE: Memory leak - ND6/W2KSP3
We are a single server 140 user shop. Here are some statistics from when we were running 6.0 under NT4 with 512 MB of RAM and McAfee GroupShield 5.21. 512 MB worked under Domino 5 until we upgraded to 6 and installed GroupShield 5.21, and the RAM usage skyrocketed. Here are statistics from that time:
4:24 PM … Shortly after server restart
total: 604MB (“Mem usage” at bottom of taskman)
nserver 69MB
nupdate 48MB
nrouter 42MB
8:24 AM … Next morning when server stopped responding
total: 646MB
nserver 176MB
nupdate 161MB
nrouter 89MB
Now after this point we did everyhing we could think of to solve the problem. Lotus support said we didn’t have anough RAM. So we upgraded to 1GB, formatted the OS partition and installed Windows 2000 Server SP4, upgraded to 6.02CF1, removed GroupShield Domino, stopped using the blackhole feature and outsourced all of our virus/spam scanning. The mem usage numbers are different (higher, probably due to Win2k vs NT4) but we are still having the same problem.
Can anyone explain this one: Look at the numbers I gave above. The differences between the “total” memory used at the two times is smaller than the difference of nrouter, nserver, and nupdate combined. What gives? I don’t have exact numbers for our current config, but they’re something like 890 MB when it crashes, but if you add up nrouter, nserver, and nupdate they total 1200MB alone, without including any other tasks in taskman. Can anyone tell me what this means?
Subject: RE: Memory leak - ND6/W2KSP3
but they’re something like 890 MB when it crashes, but if you add up
nrouter, nserver, and nupdate they total 1200MB alone, without including
any other tasks in taskman. Can anyone tell me what this means?
Taskman lies.
here is an example from live system (replication hub on high load)
Subject: RE: Memory leak - ND6/W2KSP3
Perhaps this can explain it…
Subject: Memory leak - ND6/W2KSP3
Same is happen with our server with the same files. Once nupdate gets around the 320MB mark I start seeing corrupt database files -usually starts with the log file, but I’ve also seen it corrupt mail files. I’ve been forced into restarting the server every couple days to avoid having them grow to big and corrupt things.
Subject: Memory leak - ND6/W2KSP3
is there really not anybody who has solved this issue in their environment?
Subject: RE: Memory leak - ND6/W2KSP3
my problem is more from the smtp, so I do the following two solutions:
Solution 1:
add a program document which do
nserver -c “tell smpt q”
and schedule this to run every 12 hours, then follow that add another below to run after 2 minutes after the first.
nserver -c “load smtp”
in case this does not load smtp (if smpt takes more than 2 minutes to quit), I add another one
nserver -c “load smtp”
if smpt has already loaded, then just give you an error message.
I think you can do this for other tasks.
Solution II:
I also added a monitor to monitor the mem.free, then if the free momory is under xxxx bytes, the event will trigger a program
nserver -c “restart server”
this will shut down and wait for 10 seconds to restart the domaio sever.
This works perfect.
Let me know the results if you guys try them.
Greg
Subject: RE: Memory leak - ND6/W2KSP3
I have a small twist on this that no one has mentioned, but it may help find the cause…
I have been running Domino 6.5 for at least 6 months now, with no problems at all. Last night I installed Veritas BackupExec (on a different server) and then Veritas Remote Agent on my Domino server. Ever since that time, I’ve had my server hang 3 times and require a server reboot - and it seemed that the nserver and nupdate were both using about 400MB of RAM, plus the nserver was at 25% CPU with virtually no load (at night).
So - it seems (at least to me) that this issue may have been caused by the Veritas agent? Does this ring a bell with anyone?
My only other explanation is that the act of backing up the server using the Veritas Domino Agent drove the memory and CPU usage up faster than it normally would, and so this caused the crashing to become noticable. I think that this may be the problem, since all of my crashed happened during or after attempts to backup the server. I noticed that the mem and CPU usage did not return to normal after the backup was finished, and the server crashed a couple of hours later. Previously I was stopping the server and taking a copy of all dbs each night (which would also explain why I haven’t seen this problem all this time).
Reading it over, I think this explains things…it wasn’t likely the Veritas agent that caused the problem, it’s just what caused me to start noticing the problem.
So…I guess the only real workaround seems to be to schedule jobs to restart the server from time to time? Geez…
Subject: RE: Memory leak - ND6/W2KSP3
Has anyone been able to resolve the issue. It seems that the nserver task (and others that constantly run) continue to build the mem usage while they are accessed but not release it when idle.
Any solutions ?
Subject: Steady State ?
I believe some tasks do not release memory. They ramp up to a point where they level off. I believe nserver and nhttp behave this way.As far as nadminp goes, I added a program document with…
Program name = “nserver” and…
Command line = -c “restart task adminp”
I set the schedule for once a week on Sunday morning at 5:00am. This let’s the task, “Rename in Reader/Author fields” complete (which usually skyrockets memory usage) if a bunch of people have been renamed or removed from the domino directory during previous weeks.
Subject: RE: Steady State ?
All of mine get up to a point then basically freeze the clients use of the databases. And the minimise and maximise the window does fix this however I don’t intend on doing this the 2 or 3 times per day.
Subject: Dan Ring & Backup Software
We had a very similar issue with a client.We tried all sorts with and without the assistance of IBM but to no avail.
Eventually as part of general maintenance the backup software was upgraded.
Both the generic arcserve windows agent and the lotus notes agent were loaded on the boxes. Those were version9 (sorry cant provide detailed build numbers since its gone now).
Build now in use after upgrade is 11.0.2670.19.
This upgrade immediately fixed the memory and degrading performance issues.
CA had not heard of this before.
HTH.
Subject: Leak memory
I set:Server_Name_Lookup_NoUpdate=1
FT_LIBNAME=ftgtr40.dll
ConstrainedSHM=1
ConstrainedSHMSizeMB=2048
PercentAvailSysResources=80
in Notes.ini file and try to run:
nservice.exe -jc -c
but it still not working… the Memory usage still keep increasing gradually…
Any suggestion?