We are running some C API code called from Java in the HTTP process and, sometimes, we are getting an “Insufficient Memory.” message, followed by a server crash. “Sometimes” means that it is not consistent, as the same operations in our code can work without any problem, and crash on the first run, the 10th or never… We tracked all our memory allocation/deallocation, but we seem to be good. I believe that there is a side effect of something else there.
What tools and options can we use to track the domino server memory? Moreover, how can we analyze the .dmp file? LND says it does not recognize the format, while Visual studio just shows a summary. And we don’t have any .pdb to get access to the symbols.
Any help is welcome!
Subject: Open a PMR and at same time…
Send me an e-mail with the nsds from a few crashes…
jpaganet@us.ibm.com
Subject: AddInIdle
Okay, that sounds like it’d be a good idea, and we do have some useful “pulse” moments during processing that I think would fit for that. One question, though: does that function still apply in our HTTP-service context, since it’s not a “proper” add-in running as its own task?
Subject: More info
It’s specifically C code called via JNI on the OSGi side of HTTP, so it’s not running within an Agent as such. We don’t have any calls to AddInIdle. It does seem to be cropping up from the C side - it’s not an OutOfMemoryException or anything apparently heap-related, but instead an “Insufficient memory.” error condition bubbling up from the lower levels. Unfortunately, the exception handler in our Java code doesn’t output the status number, so I can’t say yet whether it’s specifically ERR_MEMORY or ERR_SERVER_MEMORY, since the text is the same. In any event, it’s not one of the various other similar messages that contain more information.
Subject: These may be too simple for you …
Is the C API code calling AddInIdle(); occasionally? That call does some kind of memory trash collection in the past. Without it API programs on-server have always leaked memory.
It’s the C API code that’s reporting the error & not Java, right (or maybe y’can’t tell with an immediate crash)? If it turns out to be Java, we’ve got an issue with the Java gc() not running, and Java agents clog up the Java heap on server. So we’re in the habit of running gc() explicitly inside some long-running loops.
Subject: ok, some indicators that it’s AddInIdle() …
If you can run something like perfmon that’s watching memory consumption by your application, the memory consumption will follow a stair-step pattern. After “x” number of calls, the memory will go up, and it won’t go down.
Or, it’s a simple thing to call AddInIdle(); right at the end (or start) of a call into the API. It requires nothing more than including one of the .h files. That’s normally what I do to see if the memory settles after that point.