Fixup on my mail database instantly crashed Domino

I had been using the OpenNTF template on my mail database, but replaced the design back to the stock R6 mail template and then began getting “Illegal Circular Use OpenNTF<something_illegible>” whenever I tried to open any of my folders. Another forum post I found suggested running compact, fixup and updall on databases exhibiting these “Illegal Circular Use” errors. I ran compact and it didn’t seem to change anything, and than ran fixup and it instantly crashed Domino. I could not get any respone at all out of the server, or using the “server -c ‘some command’” commands. Every domino backend process was completely unresponsive, so cringingly I had to “kill -9” them. Thankfully everything came back to life just fine at startup, with many consistency checks getting run.

Here’s the snippet from my console log.


Load Fixup “mail\neal_howard.nsf” >/notes/notesdata/rem93207.con

03/13/2003 10:25:08 Remote console command issued by DOMINO ADMIN/CWF: Load

Fixup “mail\neal_howard.nsf”

03/13/2003 10:25:08 Closed session for DOMINO ADMIN/CWF Databases accessed:

0 Documents read: 0 Documents written: 0

03/13/2003 10:25:12 Database Fixup: Started

Stack base = 0x2ff219d0, Stack size = 9880 bytes

Fatal Error signal = 0x00000004 PID/TID/K-TID = 22148/1/112921

Domino Halted. Type ‘quit’ to Terminate/Cleanup


Of course I could not type ‘quit’ anywhere since the server would not read from the command line anymore. Duh.

After everything was restarted and all the consistency checks completed, I replaced my database design back to the OpenNTF template and everything seems back to normal, but is OpenNTF a “Hotel California” thing, one you check in, you can never leave?

Subject: Fixup on my mail database instantly crashed Domino

signal 0x00000004 is SIGILL, an illegal instruction. This usually occurs because of a stack overwrite of a function header. When that function tries to return to the caller it instead tries to return to a bad address and crashes.

Once Domino crashes, the server console is no longer available. The “Domino Halted. Type ‘quit’ to Terminate/Cleanup” prompt is posted directly to the terminal window and the ‘quit’ response is then read directly from the terminal. It will then properly terminate all processes and do any other necessary cleanup.

If you choose not to (or for some reason cannot) type ‘quit’, then you can also use “nsd -kill”. Run this from your data directory and it will cleanup everything. Using kill -9 will only terminate the processes (and is painfully slow as well!)

Subject: RE: Fixup on my mail database instantly crashed Domino

I couldn’t type “quit” because the terminal would no longer accept any character input. It was locked up solid after the crash. I normally run the server via a nohupped script from cron and send it’s output to a textfile since I have to restart it every night or risk a particularly nasty memory leak that consumes all memory and swapfile space, rendering the whole machine in an almost-crashed state if I leave it running more than a few days at a time. On this particular day, I was running the server directly on a dedicated IBM serial terminal session, since I had already manually stopped and restarted it once.

I didn’t know about the “nsd -kill” command.

Thanks!

Subject: RE: Fixup on my mail database instantly crashed Domino

You can also use the console logging feature starting in 6.0 as an alternative to redirecting the output to a file.