Server crush on updall

Hi,

I have found that my domino crashed with follwing error:

INFO: New files added/deleted to/from directory ‘/data/local/notesdata’

INFO: Generating binary list file ./nsd.notes/nsd_Release_8.5_cache.ins.lst

couldn’t open file

INFO: Generating cache file ./nsd.notes/nsd_Release_8.5_cache.ins

Server Name : domino

Script Version : /data/opt/ibm/lotus/notes/latest/linux/nsd.sh 8.5.00.8318

Notes Version : Release 8.5 December 05, 2008 (32-bit)

Notes Base : 8.5

Data Dir : /data/local/notesdata

Notes Exec Dir : /data/opt/ibm/lotus/notes/latest/linux

Search Path : /data/opt/ibm/lotus/notes/latest/linux /data/local/notesdata /data/opt/ibm/lotus/notes/latest/linux/res/C /data/opt/ibm/lotus/notesapi

Library Path :

Debugger : /usr/bin/gdb

Debugger Version: GDB; openSUSE 11.1

MEMCHECK Version: Memcheck Version: 8.5.00.8318

Script Dir : /data/opt/ibm/lotus/notes/latest/linux

Host Info : Linux lotus 2.6.27.7-9-default #1 SMP 2008-12-04 18:10:04 +0100 x86_64 x86_64 x86_64 GNU/Linux

User : notes (notes)

Date : Wed May 27 02:08:56 CEST 2009

Last file mod : Fri Nov 14 21:25:15 2008

Input arguments : -batch -crashpid 20392 -crashtid 4120988480 -wrapper

root R 1 0

root ? . 7893 1 SCREEN -S Lotus -m -d su -l notes -c cd /data/notesdata; /opt/ibm/lotus/bin/server

root R … 7894 7893 su -l notes -c cd /data/notesdata; /opt/ibm/lotus/bin/server

notes R … 7895 7894 -bash -c cd /data/notesdata; /opt/ibm/lotus/bin/server

notes R … 7920 7895 /bin/sh /opt/ibm/lotus/bin/server

notes R … 7992 7920 /data/opt/ibm/lotus/notes/latest/linux/server

notes R … 16539 7992 /data/opt/ibm/lotus/notes/latest/linux/event

notes R … 17200 7992 /data/opt/ibm/lotus/notes/latest/linux/replica

notes R … 17201 7992 /data/opt/ibm/lotus/notes/latest/linux/router

notes R … 17204 7992 /data/opt/ibm/lotus/notes/latest/linux/update

notes R … 17206 7992 /data/opt/ibm/lotus/notes/latest/linux/amgr -s

notes R … 17789 17206 /data/opt/ibm/lotus/notes/latest/linux/amgr -e 1

notes R … 19721 17206 /data/opt/ibm/lotus/notes/latest/linux/amgr -e 2

notes R … 17208 7992 /data/opt/ibm/lotus/notes/latest/linux/adminp

notes R … 17223 7992 /data/opt/ibm/lotus/notes/latest/linux/sched

notes R … 17227 7992 /data/opt/ibm/lotus/notes/latest/linux/calconn

notes R … 17282 7992 /data/opt/ibm/lotus/notes/latest/linux/rnrmgr

notes R … 17314 7992 /data/opt/ibm/lotus/notes/latest/linux/http

notes R … 17318 7992 /data/opt/ibm/lotus/notes/latest/linux/ldap

notes R … 17330 7992 /data/opt/ibm/lotus/notes/latest/linux/pop3

notes R … 17331 7992 /data/opt/ibm/lotus/notes/latest/linux/imap

notes R … 17332 7992 /data/opt/ibm/lotus/notes/latest/linux/smtp

notes R … 20392 7992 /data/opt/ibm/lotus/notes/latest/linux/updall

<@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@>

Section: Notes Process Info (Wed May 27 02:08:56 CEST 2009)

<@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@>

<@@ Notes Process Info → Call Stacks for process /data/opt/ibm/lotus/notes/latest/linux/updall @@>

prog: updall pid(s) : 20392 (Wed May 27 02:08:56 CEST 2009)

======== Attaching to /data/opt/ibm/lotus/notes/latest/linux/updall 20392 (Wed May 27 02:08:56 CEST 2009) =============

Attaching to program: /proc/20392/exe, process 20392

0xffffe430 in __kernel_vsyscall ()

Thread 1 (Thread 0xf5a14b40 (LWP 20392)):

#0 0xffffe430 in __kernel_vsyscall ()

#1 0xf5efb3ad in select () from /lib/libc.so.6

#2 0xf62624cc in FRDoSleep ()

from /data/opt/ibm/lotus/notes/latest/linux/libnotes.so

#3 0xf62636c0 in OSRunExternalScript ()

from /data/opt/ibm/lotus/notes/latest/linux/libnotes.so

#4 0xf6264cda in OSFaultCleanupExt ()

from /data/opt/ibm/lotus/notes/latest/linux/libnotes.so

#5 0xf6264dc4 in OSFaultCleanup ()

from /data/opt/ibm/lotus/notes/latest/linux/libnotes.so

#6 0xf6234e30 in fatal_error ()

from /data/opt/ibm/lotus/notes/latest/linux/libnotes.so

#7

#8 0xcf5ba21c in gtrFreePosBroker ()

from /data/opt/ibm/lotus/notes/latest/linux/libftgtr40.so

#9 0xcf53eff7 in gtr_IDXclose ()

from /data/opt/ibm/lotus/notes/latest/linux/libftgtr40.so

#10 0xcf57259f in gtr_MergePatt ()

from /data/opt/ibm/lotus/notes/latest/linux/libftgtr40.so

#11 0xcf5731fa in gtrMergeWorkFiles ()

from /data/opt/ibm/lotus/notes/latest/linux/libftgtr40.so

#12 0xcf57333e in gtrMergeWorkFiles ()

from /data/opt/ibm/lotus/notes/latest/linux/libftgtr40.so

#13 0xcf57794f in GTR_createIndexLast ()

from /data/opt/ibm/lotus/notes/latest/linux/libftgtr40.so

#14 0xcf5984c3 in GTRcreateIndexLast ()

from /data/opt/ibm/lotus/notes/latest/linux/libftgtr40.so

#15 0xcf525823 in cGTRio::Close() ()

from /data/opt/ibm/lotus/notes/latest/linux/libftgtr40.so

#16 0xcf524ec0 in FTGCommitIndexes(FTG_CTX*) ()

from /data/opt/ibm/lotus/notes/latest/linux/libftgtr40.so

#17 0xcf5181f7 in FTGIndex ()

from /data/opt/ibm/lotus/notes/latest/linux/libftgtr40.so

#18 0xf7065398 in FTCallIndex ()

from /data/opt/ibm/lotus/notes/latest/linux/libnotes.so

#19 0xf7066743 in FTIndexExt2 ()

from /data/opt/ibm/lotus/notes/latest/linux/libnotes.so

#20 0xf7066cd0 in FTIndexExt ()

from /data/opt/ibm/lotus/notes/latest/linux/libnotes.so

#21 0xf7066d14 in FTIndex ()

from /data/opt/ibm/lotus/notes/latest/linux/libnotes.so

#22 0x0804c4c8 in UpdateFullTextIndex ()

#23 0x0804dc34 in UpdateCollections ()

#24 0x0804dfdb in UpdateAllFiles ()

#25 0x0804be26 in Updall ()

#26 0x0804c198 in AddInMain ()

#27 0x080545b5 in NotesMain ()

#28 0x080546fa in main ()

eax 0xfffffdfe -514

ecx 0x0 0

edx 0x0 0

ebx 0x0 0

esp 0xfffc11e0 0xfffc11e0

ebp 0xfffc11e0 0xfffc11e0

esi 0x0 0

edi 0xfffc122c -257492

eip 0xffffe430 0xffffe430 <__kernel_vsyscall+16>

eflags 0x296 [ PF AF SF IF ]

cs 0x23 35

ss 0x2b 43

ds 0x2b 43

es 0x2b 43

fs 0x0 0

gs 0x63 99

quit

The program is running. Quit anyway (and detach it)? (y or n) [answered Y; input not from terminal]

Detaching from program: /proc/20392/exe, process 20392

<@@ Notes Process Info → Call Stacks for process /data/opt/ibm/lotus/notes/latest/linux/server @@>

prog: server pid(s) : 7992 (Wed May 27 02:08:57 CEST 2009)

======== Attaching to /data/opt/ibm/lotus/notes/latest/linux/server 7992 (Wed May 27 02:08:57 CEST 2009) =============

Attaching to program: /proc/7992/exe, process 7992

0xffffe430 in __kernel_vsyscall ()

Do you have a any suggestion what I must do with it?

Regards

Leszek G.

Subject: Do you know what database updall was Full Text Indexing?

I would remove the Full Text Index and let it get rebuilt from scratch… If the problem persists open a PMR with Lotus Support and seed them the database… A work around until they fix would be to disable FT Index on this database…

jpaganet@us.ibm.com

Subject: Domino server intermittently crashes during full text indexing on GTR_PBUF

Suppose to be solved in 8.0

Link

Date

Topic

Frequently asked question - How to analyze Notes/Domino NSDs? (Jean-Yves Riverin)

You should call IBM support

JYR