I’d like to share some frustration and find out if other people see this as well, besides that Lotus Notes is fat slow client and we all know this, and has its own issues which people who needs to support this product trying to explain end users and management that it is learning curve rather then Notes problem, there are things that you cannot explain to other and still tell them it’s normal and how it suppose to be. To the point, we are 8.5.1 domino 8.5.1 Notes clinet most mobile users workking remotly.Problem we are facing is that although we have 4x full T1s dedicated for Notes traffic we are experiancing unaceptable slowness, after engaging our ISP to tell us why our 4 T1s showing only 0.3 Mbps upload when testing from internal network we were given spreadsheet with external IPs connecting to our mail server on notes ports which are eating up all the bandwidth, as you can guess after comparing those public IPs connecting to our Domino mail server with external Notes client IPs we found in Domino Admin console there is nothing suspecious, those were all our Notes clinet, users trying to work… question is how it is possible that 43-40 Notes clinets can beat up 6 Mbps pipe so badly that it is unusable?!
we open PMR with IBM to see why single Notes client is taking so much bandwidth and if there are any tweeks to minimize it, besides getting a lecture about notes client architecture and that we cannot comapre anything else with it no help.
It is sad to share this info but becasue of that, no only becasue of that but because this is what pushed decision over the edge, by the end year we are migrating to exchange 2010 and outlook 2010.
Does anyone is facing same problem when external notes client are eating all you bandwidth to the point where it is unusable?
also as a side note this is regardless if external client is using local replica or mail on server, as long Notes is open and there is connection to server it is still doing it. If it is put in Island mode bandwidth is released.
Subject: MTU issue?
Probably not but I’ve resolved some latency and connection issues in the past that were MTU related.
You may want to take a look at this posting: http://www-10.lotus.com/ldd/nd6forum.nsf/55c38d716d632d9b8525689b005ba1c0/0908a7a9ac788451852572ac006f1ba6?OpenDocument&Highlight=0,mtu
Subject: Here’s Where I Would Start
I certainly haven’t experienced the same connectivity/bandwidth utilization issues that you have seen, but if I were faced with this situation and the information that the ISP is giving you, this is how I would start.
First, look at the bandwidth utilization on the Domino Server; look at how much of the server’s network connection is being used. How does this compare to what the ISP is telling you when you match up Mbps (not percent utilization)?
Second, are both the Notes client and Domino Server(s) network config set to compress data? As an aside, what version Notes and Domino are we talking about?
Third, given that going into island mode at the Notes client reduces network connectivity (as one would expect) but that using a local replica has the same impact as using a server replica, check out what the Notes client is actually doing. What databases are local that are being replicated? And take a look at the replication behavior of the Notes client. I would be suspicious of something on the Notes client that is replicating too much data too often. There could be lots of reasons for this, but you should be able to narrow what what (if anything) is contributing to this by evaluating the Notes client log for replication events. There may be databases that are being updated too often and replicated too often, causing the Notes client to be constantly busy replicating with the Domino Server.
This doesn’t really sound like an issue with what the user is doing with your Domino Server(s) but rather an issue with inefficient or continuous replication by the Notes client with the Domino Server(s).
Subject: we tried pretty much everything
First, look at the bandwidth utilization on the Domino Server
They do match up, we do not see problem internally because we are on 1Gb network with, problem is that 4 T1s are giving only 6 Mbps and this is not enough even for as little as 40 remote folks and it is not individual client or clients, it is real easy to see the difference when we have 20 vs 40 or more Lotus Notes clients
Second, are both the Notes client and Domino Server(s) network config set to compress data? As an aside, what version Notes and Domino are we talking about?
Yes they all configured with data compression and encription, and we use lastest versions 8.5.1 FP2 on Domino and 8.5.1 clients
Third, … but that using a local replica has the same impact as using a server replica, check out what the Notes client is actually doing.
This is real funny actually we are not using Domino/Notes for anything other then vanila email/calendar and there is no other databases replication going on, all clients are configured to replicated on schedule every 30 minutes via policy that cannot be modified by user, short of long even there is no replication in place Notes client is still “pilling” something from Domino and reason I use “pull” word here is becasue we see mostly issue with upload from internal network side which means other side is downloading. Make no sense but this is how it is.
This doesn’t really sound like an issue with what the user is doing with your Domino Server(s) but rather an issue with inefficient or continuous replication by the Notes client with the Domino
inefficient??? it is straight forward install other then Domino is in cluster but that should make no difference since Notes client are connecting to one of the nodes only at the time and other node is in remote location.
again, we had PMR open with IBM and did not get any help all support was trying to tell us is that NRPC protocol is not best and efficient which BTW is not best way to promote a software.
don’t get me wrong I have beed working and supporing Domino/Notes for last 8 years but it is so frustrating when you have to work with something that does not work well at least in this particular situation when we have folks working remotly.
Subject: Not my area but…
Hi,
I normally don’t handle this area, so forgive me if I am asking something silly.
The only obvious thing I can think offhand is that are the users using the local copy of the mail file or the server copy?
Also if you run the client in basic mode (command line argument -sa ), do you get the same results? If not then you have narrowed it down a bit.
I’ll flag the thread for an engineer who covers these areas.
Subject: both local and server replica same result
also to answer your question regarding basic Notes we did not try that but that doesn’t matter as this is not a solution even if it makes a difference, we want to use Standard client.
Subject: Re: Basic client.
Hi,
OK it may not be a solution can use, but would rule out a lot of areas of the client as being the cause.
Subject: Any Luck Yet?
Tim, by “inefficient” replication, I meant merely that something is causing replication to occur when it shouldn’t need to. There are lots of potential reasons for this to occur but it isn’t clear that this is really the case.
If you are seeing continuous or near-continuous pulling by the Notes client from the Domino Server, then I think it is important to identify what the Notes client is doing when it is pulling that data. Is it a scheduled replication event from the Notes client that is running continuously? If so, is it actually replicating data? If so, what is it that is causing the Notes client to continually think that the Domino Server has new or modified data that needs to be pulled down to the Notes client.
The fact that your bandwidth is fully utilized by your 40 Notes client might be a symptom of changes being made by the Domino Server to the user’s mailfile, causing the Notes client to continuously (or nearly) replicate with the Domino Server. It is not normal for a remote Notes client to be continuously replicating with a Domino Server if all it is doing is replicating a mailfile and the person isn’t receiving hundreds of messages an hour. Perhaps there is something other than replication that is causing this Notes client communication with the Domino Server, but I don’t know what that could be, to be honest.
Can you get a hold of the replication logs from the Notes clients and see what is going on there? I would be suspicious of long and frequent replication if you see that.
Subject: We don’t see this either…
-
In addition to William’s excellent suggestions you might consider doing a packet capture on the server to see exactly what each IP is doing. Maybe only one or two clients are misconfigured, and without any manner of traffic shaping those few could soak up an impressive amount of bandwidth, such that the majority remaining have little to go around.
-
Other tools may do this same thing. I just like packet captures. If the data isn’t encrypted (unlikely with Domino) then you have the added advantage of seeing what’s going where as well as how much of it.
Hope this helps…
Subject: others see this as well
I’m not the only one, I found this on Domino 8 formu but no responses
http://www-10.lotus.com/ldd/nd8forum.nsf/5f27803bba85d8e285256bf10054620d/e4f3baa66ec4613b852573480024e619?OpenDocument&Highlight=0,150mb
this is strange issue which you won’t spot until you have enough people to completly kill your asscees to the internet or situation where ISP is limiting your bandwidth or if you do close monitoring of your pipes
Subject: tips analyze the problem
We do not have such problems on our site, so this is problem is special in your environment.
If i had to troubleshoot such a problem. I will use Wireshark on one client to analyze the network traffic. If this does not explain the problem you can activate a trace function in your Notes client to see which nrpc functions are called by your client. See http://www-01.ibm.com/support/docview.wss?rs=463&uid=swg21239553 for informations how to activate this trace. With this tools it will be very easy to find what is causing your bandwithproblems.
Ralf M Petter