Do we need to recycle Notes objects in Java agents or web services?

HCL Domino Domino Forum
1

Hi,

I’m a long-time Notes/Domino developer… but I just started using Java in Notes/Domino.

So far, I did a Java web service as well as a couple of Java agents. Everything’s working fine.

However, I just ran into this technote by accident:

http://www-01.ibm.com/support/docview.wss?uid=swg21097861

I thought I might have forgotten something important, because I didn’t recycle my Notes objects.

However, I read this technote three times and I’m still not sure I understand that line correctly:

“When using objects in an agent, all objects (both Java and C++) are destroyed when the agent ends. When using servlets, .jsp’s, or standalone applications, recycle must be used since Domino will never clean up these backend objects.”

1- Does that mean that when using an agent, I don’t need to recycle my Notes objects?

2- Do I need to “null” my Java objects (both Notes objects and Java regular objects?) or are they automatically nulled when the agent exit (and then available for GC)?

3- Are Web Services acting like Agents? Or are they considered in the same group as servlets, .jsp’s, …?

Thanks a lot!

2

Subject: answers for 1 and 2

To the best of my knowledge…

1- Does that mean that when using an agent, I don’t need to recycle my Notes objects?

=> True in principle. But you may still want to recyle, in order to free memory while the agent is running. This may even be necessary if the agent processes large amounts of Notes objects in one run.

2- Do I need to “null” my Java objects (both Notes objects and Java regular objects?) or are they automatically nulled when the agent exit (and then available for GC)?

=> You don’t need to ‘null’ them. They are nulled automatically as soon as they go out of scope. That is, most of them (those not having global scope) get nulled even before the agent terminates.

As to your third question, I assume Web services act like agents. At least that’s what Designer help suggests. Quote: “The compiled Web service, like an agent, is a stand-alone program in a Domino database.”

3

Subject: Web services are basically agents.

You do not need to recycle the web service. Only notes objects which were not created off the session of the agent.

As a general rule I always clean up (except the session in the agent). It is good practice.

4

Subject: That’s what I thought… but…

Hi Jochen,

What you’re saying seems to confirm what I read about on this forum:

http://www-10.lotus.com/ldd/46dom.nsf/7d6a87824e2f09768525655b0050f2f2/4600bba5f42ed834852569290053739e?OpenDocument

http://www-10.lotus.com/ldd/nd6forum.nsf/55c38d716d632d9b8525689b005ba1c0/ddbeddd7221801d785256ebb006afe59?OpenDocument

http://www-10.lotus.com/ldd/nd6forum.nsf/55c38d716d632d9b8525689b005ba1c0/3028acdd0eb5e0c785256ed200791765?OpenDocument

Those posts are extremely old so I wanted to make sure it was still working like that.

And since those posts are old, they didn’t mention web service providers. I believe, like you, that they are the same thing as agents, but I wanted to have a confirmation.

I did a test, I called my web service like a hundred times (maybe even more) and the nhttp task on the server went from 31 megs to 99 megs of RAM. I waited a couple of hours, and it was still 99 megs. This morning it was 70 megs.

I wasn’t doing any recycle. How long should it take before Domino recovers the memory? That’s why I’m wondering if web services are the same as agents.

Thanks for your answer!

5

Subject: I’ve seen similar behaviour with Java agents

Admittedly, my own experience with that kind of stuff is about 3-4 years old (Notes 6.5 and 7, that was). But I suspect it still applies:

The reason for the Agent Manager eating up memory turned out to be the class loader of the JVM that comes with Notes/Domino. Apparently, classes are never dumped, not even after the last agent using them has terminated. One has to kill amgr to free the memory occupied by Java classes – including any static objects (!!)

(Just as a side note, not related to your problem: Things are even worse with Java applets in the Notes client. Every class used by an applet is re-loaded from scratch with every instance of the applet. Again, no dumping classes when an applet terminates. Load a larger applet multiple consecutive times, and the Notes client will go out of memory real soon. That’s why Java applets are almost unusable in the Notes client, unless one uses a common workaround by specifying the classpath with the help of notes.ini/JavaUserClasses, which forces classes to be at least re-used. Fortunately, the same is not necessary with amgr.)

6

Subject: So to recycle wouldn’t correct this?

“The reason for the Agent Manager eating up memory turned out to be the class loader of the JVM that comes with Notes/Domino. Apparently, classes are never dumped, not even after the last agent using them has terminated. One has to kill amgr to free the memory occupied by Java classes – including any static objects (!!)”

Are you implying that even If I did all the appropriate recycle, I would still experience the same results? (increase of memory?)

Thanks again!

7

Subject: two different animals

As discussed and described by you, recycling is for reclaiming resources occupied by Domino objects (Database, Document, …)

My point was the Java classes including static objects. Domino’s JVM appears to keep them in memory until it eventually decides to disard them. From what I can see – similar to what you seem to experience in terms of memory consumption – I conclude that they are kept in memory for very long, if not until the relevant Domino server task (be it amgr or http) is shut down. I have never seen any documentation on how this behaviour can be influenced by a developer.

8

Subject: I did some more tests…

Hi again,

I did some more tests using an agent this time… just for fun.

The nhttp process takes around 140megs of RAM when it is first launched on our dev server.

I ran this agent once (without recycle):

      Session session = getSession();

      PrintWriter pw = getAgentOutput();

      

      Database dbNames = session.getDatabase("", "names");

      View viewNames = dbNames.getView("($VIMPeople)");

      Document docNames = viewNames.getFirstDocument();

      while (docNames != null) {

    	  pw.println(docNames.getItemValueString("FullName") + "<br>");

    	  docNames = viewNames.getNextDocument(docNames);

      }

The memory increased from 142 megs to 192 megs!

I quit and load the http task again on the server console and then I ran the same agent again. The memory increased again from 136 megs to 191 megs.

I quit and load the http task again and ran the following agent (with recycle):

      Session session = getSession();

      PrintWriter pw = getAgentOutput();

      

      Database dbNames = session.getDatabase("", "names");

      View viewNames = dbNames.getView("($VIMPeople)");

      Document docNames = viewNames.getFirstDocument();

      Document nextDocNames=null;

      while (docNames != null) {

    	  pw.println(docNames.getItemValueString("FullName") + "<br>");

    	  nextDocNames = viewNames.getNextDocument(docNames);

    	  docNames.recycle();

    	  docNames = nextDocNames;

      }

The memory increased from 138 megs to 144 megs.

I quit and load the http task again and ran the same again. The memory increased from 137 to 143 megs.

It’s much better… and I can understand why.

But why isn’t the memory going back to the number of megs if was before I ran the agent (when the agent is completed)? I ran those agents from a web browser, so the amgr task isn’t used (I tried quit and load on the amgr task just for fun, but as I expected, it didn’t change the nhttp task memory).

I’m confused! :slight_smile:

9

Subject: pretty much in sync with my own experience

Recycling can substantially reduce the amount of additional memory occupied by a Java agent. But it cannot reclaim entirely the memory occupied by the agent. Reasons for this are various, I assume. One is Java classes (see my previous responses to this thread), another has been named by Bjorn (see http://www-10.lotus.com/ldd/nd85forum.nsf/ShowMyTopicsAllFlatweb/5a525b0bedd7fba28525779000761258?OpenDocument).

I don’t find your results confusing, after all.

10

Subject: now in LotusScript

The same agent in LotusScript:Sub Initialize

Dim session As New NotesSession

Dim dbNames As NotesDatabase

Dim viewNames As NotesView

Dim docNames As NotesDocument



Set dbNames = session.Getdatabase("", "names.nsf")

Set viewNames = dbNames.Getview("($VIMPeople)")

Set docNames = viewNames.Getfirstdocument()

Do While Not docNames Is Nothing

	Print docNames.Getitemvalue("FullName")(0) + "<br>"

	Set docNames = viewNames.Getnextdocument(docNames)

Loop

End Sub

The nhttp task went from 137 megs to 141 megs on the first try and then from 138 megs to 141 megs on the second try (I restarted http between the tests).

It’s quite similar to the Java agent WITH recycle.

Again, the nhttp process didn’t recover the memory after the agent completed (it’s still 141 megs).

Is it just because it takes time before Domino recover the memory?

11

Subject: It doesnt reclaim the memory

…at least if you look at it from the OS level. As I understand it, this is because Domino manages its own memory, and will keep what it has used, just in case (somebody probably has a better explanation on it than me)

For a good (and recent) article on Domino and Java, read this: http://www-10.lotus.com/ldd/ddwiki.nsf/dx/Java_memory_management_in_NotesDomino

12

Subject: Web Service example

Thanks Bjorn for the link… it was an interesting read. I think I have a better idea on how it works with agents now. Thanks Jochen and Simon for your precisions.

If you don’t mind, I’d like to ask a web service related question now. Take the following web service:

import lotus.domino.*;

public class JavaWithRecycle {

private Database _dbNames=null;



public JavaWithRecycle() {

	try {

		Session session = WebServiceBase.getCurrentSession();

		_dbNames = session.getDatabase("", "names");

	} catch(Exception e) {

		e.printStackTrace();

	}

}



public String loopPeople() {

	StringBuilder sb = new StringBuilder();

	try {

		View viewNames = _dbNames.getView("($VIMPeople)");

		Document docNames = viewNames.getFirstDocument();

		Document nextDocNames=null;

		while (docNames != null) {

			sb.append(docNames.getItemValueString("FullName"));

			sb.append(",");

			nextDocNames = viewNames.getNextDocument(docNames);

			docNames.recycle();

			docNames = nextDocNames; 

		}

		viewNames.recycle();

	} catch(Exception e) {

		e.printStackTrace();

	}

	return sb.toString();

}



public String loopGroups() {

	StringBuilder sb = new StringBuilder();

	try {

		View viewNames = _dbNames.getView("($VIMGroups)");

		Document docNames = viewNames.getFirstDocument();

		Document nextDocNames=null;

		while (docNames != null) {

			sb.append(docNames.getItemValueString("ListName"));

			sb.append(",");

			nextDocNames = viewNames.getNextDocument(docNames);

			docNames.recycle();

			docNames = nextDocNames; 

		}

		viewNames.recycle();

	} catch(Exception e) {

		e.printStackTrace();

	}

	return sb.toString();

}

}

As you can see in this simple example, I did my best to recycle all Document objects in the loop as well as the View object. So this code doesn’t take much memory (as opposed to when I didn’t recycle).

My question is, where in the code should I recycle the Database object? It’s a member of the class that is initiated in the constructor and use in more than one methods. Also, should the Session object be recycled (and if so, where?)? I’ve read contradictory things about this.

It’s easy to know when a agent completes… but what about web services? If a .NET consumer create a new instance of my web service, the constructor is called and _dbnames is assigned a value. The consumer can then call some methods that uses the _dbnames value. When do we know when a web service completes?

Sorry for asking more questions, I’m just trying to apply what I just learned about Java agents to Java web services.

Your help is very appreciated.