Subject: Re: Data…
Good Afternoon Chris,
I recommend collecting Data for the Windows Crash and Performance Team to review. If not already enabled, add the following notes.ini parameters to your Domino Server so that IBM may troubleshoot this issue. These parameters will provide additional details if the problem occurs again and are good general debug parameters for many server problems.
debug_threadid=1
console_log_enabled=1
console_log_max_kbytes=153600
debug_capture_timeout=1
debug_show_timeout=1
**These parameters can be enabled on the server console while the server is running by using the “set config” command (i.e. set config debug_threadid=1). The two timeout parameters will not begin logging until the Domino Server is restarted, but the console log and thread ID are dynamic. When you shut down Domino to make sure the timeout parameters begin logging, can you also update the version of nsd running on the server, to make sure we have the best information we can get? The download link is found in this Technote below:
NOTE: If you are running 8.0.2 or above, you do not need to Update your NSD Version.
Lotus Software Knowledge Base Document
Title: Updated NSD for Domino releases
Doc #: 4013182
URL: http://www.ibm.com/support/docview.wss?rs=899&uid=swg24013182
Manual NSDs will need to be obtained from the Server. I recommend that the next time you Restart your Server, capture a Manual NSD -detach after the Server has come up. Then, let the Server run and when you see the CPU start to increase, obtain another Manual NSD -detach. Finally, when CPU is spiking, obtain multiple back-to-back Manual NSDs (at least 3).
When you run nsd manually, be sure to use the ‘-detach’ switch (do not do an NSD -kill unless absolutely necessary to bring down the Server; be sure an NSD -detach has completed running first). I have included a link to Technote 1204263 - “How to run a manual NSD for Notes/Domino on Windows” below as well as my own instructions:
Lotus Software Knowledge Base Document
Title: How to run a manual NSD for Notes/Domino on Windows
Doc #: 1204263
URL: http://www.ibm.com/support/docview.wss?rs=899&uid=swg21204263
All you do is simply use a Command Prompt on the Server, path out to the Domino\Data Directory. From there, type the path back to Domino\nsd -detach. Allow it to complete, and gather the data. See example below:
NOTE: This is an example. You will need to use your Domino Program and Data Paths For The Commands:
C:>cd c:\Lotus\Domino\Data
C:\Lotus\Domino\Data>C:\Lotus\Domino\nsd -detach
After the Manual NSDs complete, if the Server Console is responsive, run the following commands (this will give IBM some additional information - especially if the NSD does not complete successfully):
show stat
show tasks
show server
show users
After running the last of the Manual NSDs and Commands, please browse to the IBM_TECHNICAL_SUPPORT directory inside the Domino Data directory and collect the following four files:
-
nsd_w32i_servername_date@time.log (obtain all the Manual NSDs obtained)
-
console.log
-
semdebug.txt
-
sysinfo_W32I_NYMAIL02_date@time.log
NOTE: If you restart Domino before sending me these files, the console and semdebug will have been renamed with the same naming format as the nsd, except the date and time will be the time the server was last brought up PRIOR to the problem. Also, please ensure you obtain the latest Sysinfo File.
Additionally, provide a WINMSD Report and save it as a system information file, .nfo. To do this, go to Start-->Run and type in winmsd. Once this opens, go to File-->Save and save it as WinMSD.NFO (not txt).
Also collect your event viewer logs, application and system and save as .evt files. To do this, go to Start-->Control Panel-->Administration Tools-->Event Viewer. Once there, Click on Application, then Action-->Save Log File As... and save it as "App.evt". Next, once you save the Application, click on System, then Action-->Save Log File As... and save it as "Sys.evt".
Overall, the Files required are seen below:
-
nsd_w32i_servername_date@time.log
-
console.log
-
semdebug.txt
-
sysinfo_W32I_NYMAIL02_date@time.log
-
WINMSD Report
-
App.evt
-
Sys.evt
To upload the files, please open a PMR with IBM Support. Once the PMR is created, place the above 7 Files into a Zip File and go to the following web site:
http://www.ecurep.ibm.com/app/upload
and fill in the fields as seen in the screen shot below with your information and click “Continue”:
On the next screen, browse to the Zip File containing the 7 Files that you just created and submit it.
Please feel free to Reply to this with any questions or updates and I will respond as soon as I can!
Thank you and have a great day Chris!
Regards,
-Bob