Urgent,dozens of mail databases damaged!

HCL Domino Domino Forum
1

there are many fatal errors in log.nsf and ddm.nsf, such as “Database is corrupt – Cannot allocate space”;“Database open error: mail\userA: File truncated - file may have been damaged”;“Database open error: mail\userB: File modified at operating-system level while cached, please retry open.”;“Database open error: mail\userC: RRV bucket is corrupt.”;“Database collection open error: e:\Program Files\IBM\Lotus\Domino\data\mail\userD.nsf: invalid file handle”.

it happened once a month after we upgrade domino server from 6.0 to 8.0. can’nt find any solution and cause! please help,any suggestion is much appreciated!

2

Subject: Folder has been damaged - Same issue

Did anyone find a solution?We have a Domino cluster running on W2K3 VMware.

We keep seeing corrupted mailfiles, mainly damaged inbox folders.

It started when we upgraded from 7.02 to 8.01.

Now we are running 8.02FP1; still the same problem. mailfiles are accessed through http and nrpc.

fixup is useless, like updall; compact -c -D -i fails.

Always both instances of the db (2 node cluster) are corrupted.

Sometimes I can delete the inbox in the designer and do a replace design. Sometimes the instance on the other node could be fixed

Most of the time these mailfiles are overquota.

While in overquota, even the Domino processes cannot write to the mailfile, which seems very bad; this is the standard IBM overquota feature.

I can see (running unlocker on the file) that nhttp is locking the file.

Thanks for any input

3

Subject: My guess is that other programs is messing with your databases…

What kind of backup software are you using?Is the software compatible with Domino 8?

Are you using RAID or SAN?

If SAN; Is there similar problems on other servers using the same SAN?

Try to describe your environment more fully…

Ove

4

Subject: My guess is that other programs is messing with your databases…

thanks Ove.we didn’t use any backup software. just replicate some mail databases with aother domino server.

we are useing raid 5 and a hot backup.

5

Subject: And the virus scanner? - I have seen multiple server break downs in 2008 because of different versions of one.

6

Subject: Re：And the virus scanner?

the cause of virus scanner software war eliminated, we have stopped it for two months.

7

Subject: Hm. Checkdisk? Update to 8.0.2

you would most likely have to do both anyhow, if you want to go for IBM support.

Google (and Search in IBM support) for

“File truncated - file may have been damaged”

might help you, too. -

8

Subject: problem remains

We update grade server to 8.0.2, and totally move to another server(worry about hardware problem),but databases still corrut.

9

Subject: Corruption on folder

We’re having similar issues. Every day some databases get corrupted, but most of them have just one folder damaged. A 5% of the cases get $Inbox corrupted, another 5% cases have the $Alarms view corrupted, and the rest of 90% have the folder Expense Accounts (created by NEDS application) damaged.

This is the result of the fixup:

[319662:00002-00001] **** DbMarkCorrupt(Folder Expense Accounts corrupt), DB=/notesdata/dataxx_xxxx/mail1/xxxxx.nsf TID=[319662:00002-00001] File=dbfolder.c Line=550 ***

[319662:00002-00001] 28/01/2009 11:44:55 Database Fixup: Error fixing view 181730 in mail1/xxxxx.nsf: Folder has been damaged. Please close and reopen database to have it repaired.

There’s no way it can be repaired through a fixup. I can solve the problem temporarily by running a “compact -D”, but a few hours after that it gets damaged again.

This is happening after we upgrade our domino servers to 8.0.2.

More specifications:

Domino version 8.0.2HF194

Running on AIX 3 5

Still looking for a solution.

10

Subject: Igor is right. Additionally, I would check for enough disk space, check for nsd files and run fixup and updall -r

11

Subject: you are not alone

we’ve been suffering similarly with these spontaneous corruptions. Got another one last night out of nowhere.

Database collection open error. Our upgrade path was from 6.5.6 to 8.0.1 to 8.5.

Still have no idea what is causing them.

12

Subject: Re: you are not alone

Alvaro, are you running Domino on an AIX environment?