Google spider causing R6 server to crash

I have a server that every night would crash between 9 to 12 pm. We found that it crashed when Google accessed the site. The last entry would be a HTTP authentication error.

I set the robots.txt to prevent the google search and no more crashes.

The site has a JavaScript menu that allows access to every page including restricted pages (need to loging to access). The search robot seemed to try to access all pages at ones.

Just seeing if this has happened to others.

R6

W2k

Mix of domino and Macromedia pages with Flash.

Subject: Google spider causing R6 server to crash

We do have same problems here. System runs with Servlets. We get tons of errors in our own log file and the server crashes…

We’ll figure out if a spider crashes us…