Dear All,
We noticed that today morning, there an issue with our Traveler server and our users were not able to access the same. We are running Traveler 9.0.1.0 on Domino 9.0.1. There were these lines on the console:
28/10/2014 08:27:54 AM HTTP JVM: 1183155475 [Traveler-9128081248856_CN=User/OU=OrgUnit/O=Org-7c0] INFO - Traveler: IBM Notes Traveler task did not respond within the allotted time frame (905,000 milliseconds). Client request denied for CN=User/OU=OrgUnit/O=Org.
On closer inspection it was found that in the early hours of today there was a Java Exception on the server which read like the below:
[1460:0052-07F8] 28/10/2014 04:55:19 AM HTTP JVM: CLFAD0211E: Exception thrown. For more detailed information, please consult error-log-0.xml located in C:/Program Files/IBM/Lotus/Domino/data/domino/workspace/logs
[1460:004F-0900] 28/10/2014 04:55:19 AM HTTP JVM: CLFAD0211E: Exception thrown. For more detailed information, please consult error-log-0.xml located in C:/Program Files/IBM/Lotus/Domino/data/domino/workspace/logs
[1460:0052-07F8] 28/10/2014 04:55:19 AM HTTP JVM: >>>>
[1460:004F-0900] 28/10/2014 04:55:19 AM HTTP JVM: >>>>
[1460:0052-07F8] 28/10/2014 04:55:19 AM HTTP JVM: CLFAD0317E: The XPages runtime engine faced an OutOfMemoryError
[1460:004F-0900] 28/10/2014 04:55:19 AM HTTP JVM: CLFAD0317E: The XPages runtime engine faced an OutOfMemoryError
[1460:0052-07F8] 28/10/2014 04:55:19 AM HTTP JVM: CLFAD0316E: You can fix this by increasing the value of the HTTPJVMMaxHeapSize variable in notes.ini
[1460:004F-0900] 28/10/2014 04:55:19 AM HTTP JVM: CLFAD0316E: You can fix this by increasing the value of the HTTPJVMMaxHeapSize variable in notes.ini
[1460:0052-07F8] 28/10/2014 04:55:19 AM HTTP JVM: >>>>
[1460:004F-0900] 28/10/2014 04:55:19 AM HTTP JVM: >>>>
[1460:0052-07F8] 28/10/2014 04:55:19 AM HTTP JVM: CLFAD0315E: Out of memory exception occurred servicing request for: /traveler/Microsoft-Server-ActiveSync - HTTP Code: 500. For more detailed information, please consult error-log-0.xml located in C:/Program Files/IBM/Lotus/Domino/data/domino/workspace/lo
[1460:0052-07F8] 28/10/2014 04:55:19 AM HTTP JVM: s
[1460:0052-07F8] 28/10/2014 04:55:19 AM HTTP Web Server: Command Not Handled Exception [/traveler/Microsoft-Server-ActiveSync?User=xxx@yyy.net&DeviceId=ApplDLXM75JLFKYC&DeviceType=iPad&Cmd=Ping] CN=User/OU=OrgUnit/O=Org
[1460:004F-0900] 28/10/2014 04:55:19 AM HTTP JVM: CLFAD0315E: Out of memory exception occurred servicing request for: /traveler/Microsoft-Server-ActiveSync - HTTP Code: 500. For more detailed information, please consult error-log-0.xml located in C:/Program Files/IBM/Lotus/Domino/data/domino/workspace/lo
[1460:004F-0900] 28/10/2014 04:55:19 AM HTTP JVM: s
The trace-log shows this:
CLFAD0211E: Exception thrown java.lang.OutOfMemoryError: Java heap space at com.lotus.sync.servlet.TravelerServlet.getBody(TravelerServlet.java:1783) at com.lotus.sync.servlet.TravelerServlet.doRequestResponse(TravelerServlet.java:1062) at com.lotus.sync.servlet.TravelerServlet.service(TravelerServlet.java:1012) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180) at org.eclipse.equinox.http.servlet.internal.ServletRegistration.handleRequest(ServletRegistration.java:90) at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:111) at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:67) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at com.ibm.domino.xsp.adapter.osgi.OSGIModule.invokeServlet(OSGIModule.java:167) at com.ibm.domino.xsp.adapter.osgi.OSGIModule.access$0(OSGIModule.java:153) at com.ibm.domino.xsp.adapter.osgi.OSGIModule$1.invokeServlet(OSGIModule.java:134) at com.ibm.domino.xsp.adapter.osgi.AbstractOSGIModule.invokeServletWithNotesContext(AbstractOSGIModule.java:181) at com.ibm.domino.xsp.adapter.osgi.OSGIModule.doService(OSGIModule.java:128) at com.ibm.domino.xsp.adapter.osgi.OSGIService.doService(OSGIService.java:417) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:350) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:306) at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:272)
while the error-log shows the below:
CLFAD0315E: Out of memory exception occurred servicing request for: /traveler/Microsoft-Server-ActiveSync - HTTP Code: 500
We are running the server on a Windows 2012 server with 8 GB RAM and the HTTPJVMMaxHeapSize is set as 1024 MB.
As an immediate solution, we restarted the Traveler and the underlying Domino and everything is OK for now. Any insight into the issue will be greatly helpful.
Thanks in advance.