We have 2 members of a cluster running on 2 solaris boxes. one is r6.0.3 the other is 5.0.11. all works well.
We are load balancing HTTP with an ALteon switch. All works.
However, if server A crashes and is restarted, server B’s HTTP hangs until server A comes back up.
This defeats the whole point of the cluster. Any ideas?
Damian
p.s. we are looking for a freelance notes admin
inquire within…
damian@svothi.com
Subject: HTTP Failover in Cluster
Does the reverse happen? If you restart Server B, does Server A quit responding until Server B comes back up?
A couple of things come to mind:
-
Are you using two ports on each server, one for HTTP/Notes access, the other for cluster replication? If so, make sure that a) in notes.ini, the order of the ports is correct (e.g., Ports=TCPIP,TCPClusterPort, and Server_Cluster_Default_Port=TCPClusterPort), and b) if Solaris has a way of prioritizing the ports like Windows does, make sure the non-cluster port has the higher priority.
-
Also, if you do use a separate port for clustering, is that hooked to a separate switch, on a completely different subnet, one that the Alteon doesn’t know anything about?
-
Double-check the setttings in the Alteon to make sure that the Virtual IP is set to load-balance and fail over between the two real IP addresses on the appropriate ports. Sometimes if a port/IP combination goes in and out of service enough times in a given time period, the load balancer will simply turn off that port/IP combo until it is manually re-enabled.
Otherwise, just double- and triple-check your server and web conifiguration documents, and make sure everything is evened up.
Subject: HTTP Failover in Cluster
Damian, Usually you have a Virtual IP that gets balanced, and a Real IP on each server. When one server is down the other one doesn’t respond to either IP?:-jmw
Subject: RE: HTTP Failover in Cluster
Yes, we have a virtual IP setup in the switch. Plus we are using direct access mode so i can try to open either server directly.
There are 2 domino servers, A & B.
When A is rebooting… I cannot access notes or web on B… once A comes back up, B comes alive again. The same applies for the virtual IP as well, of course, because server B is simply unresponsive.
It’s acting as if the clustered server needs it’s sister to work… but that doesnt make for a useful cluster…
It’s not a switch problem. It’s a Domino clustering issue…
I could disconnect the switch and still see the same behavior.
Subject: RE: HTTP Failover in Cluster
A little more info…
the second server in the cluster stops responding to Notes and HTTP requests seconds after the other server shuts down. It starts responding again only after the other server in the cluster starts and domino starts!
D
Subject: RE: HTTP Failover in Cluster
Damian, Does the reverse apply, If B dies A is hung as well? I’m thinking you might have a hardcoded default home page pointing to the one server?:-jmw
Subject: RE: HTTP Failover in Cluster
Thanks John.
No, it appears to be one-way.
If Server B is shutdown or dies, Server A is OK.
If Server A is shutdown, Server B dies until Server A comes back up.
This is becoming a big nuisance… we have spent a year getting these clusters and Layer 4 switches all perfect, and now it’s not even working due to some Domino issue. I know we’ll figure it out, I just hope sooooooooon.
Help!
D
Subject: RE: HTTP Failover in Cluster
Damian, I experienced a similar issue that ended up being the default “home page/db” of the server. If you want to test this, switch the default home to a something basic like “/icons/abook.gif”. If your configuration is really that bad you should be able to verify within minutes, and switch back if needed? If that works, you have at least narrowed down the issue to the default db:-jmw
Subject: RE: HTTP Failover in Cluster
Hey John,
I don’t really understand how that can be the problem?
When Server B goes down, everything goes… Notes included. No web pages are accessible…
Where does the default homepage come into play?
Thanks for trying to help though… I really appreciate everyone’s efforts…
D
Subject: RE: HTTP Failover in Cluster
Damian, The example I ran accross was a default home page that was hard coded to always look to Server A, the same default on both Server A & B. Server A would work fine obviously, Server B would work only if server A was up, since the Server B http task could not execute the Lookup required to render the default page. Kind of obscure but it could explain your problem!:-jmw
Subject: RE: HTTP Failover in Cluster
Thanks John…
But, no, it appears that http goes completely dead as well as Notes. Ie. I can’t even connect to the server from a notes client.
However, once its cluster sister comes back up, everything starts working again.
C’mon peeps… some has to have seen this before! This is driving me nuts.
D
Subject: HTTP Failover in Cluster
I don’t have an answer but actually have a question. What is this ALteon switch that you’re using? We’re looking for a similar solution but found that ICM did not do what we wanted? I’d appreciated if you can direct me to more information about this ALteon product. Thanks!
BTW, how come you’re not using ICM?
Subject: RE: HTTP Failover in Cluster
IMHO, ICM rots. It has a lot of limitations, including the fact that its merely masquerading URLs. If you have non-Domino generated URLs or hardcoded URLs with the domain name, etc., you’ll have problems. We couldnt afford to go back and redesign all of our apps. Additionally, we have PERL apps and stuff that needed to be load balanced in coordination with Domino.
We needed a real L4 solution.
Alteons AD3’s are great and you can get them cheap. Check out Nortel’s site for docs and stuff. They are a little tricky to get rolling, setting up the persistence of sessions and health-checks and all that, but offer the only robust solution for SLB. You might also look at F5 and other L4 switches.
The only thing I am annoyed with re: the Alteons is the persistence of sessions across http and https… it can’t be done unless we use hash or minlosses as a metric, which doesnt afford us much control in how to balance the traffic… but this is getting off track. Just make sure your boxes are “close” in operating power 
D