HTTP Server Crash

HCL Domino Domino Forum
1

We’ve detected several crashes in our domino server causes for http task. We can’t detect any secuence or cause for that, and it can ocurrs various times during a day, or once in 2 days, etc.

Any help will be appreciated (very much).

NSD output:

0x42145d01: __nanosleep + 0x11 (1, 41f55ca4, b99c3984, b99c3984, b99c3570, 0) + 354

0x406602f1: OSRunExternalScript + 0x15d (0, 41f55ca4, b99c3984, b99c3984, 0, 0) + 1c8

0x4065f06a: OSFaultCleanup + 0x16a (0, 0, 0, 81c7e14, b, b99c3c90) + 20c

0x40640c05: fatal_error + 0x12d (b, b99c3c90, b99c3d24, b99c3d24, b99c3c90, b)

0x45c44892: panicSignalHandler + 0xea (b, b99c3c90, b99c3d24, b, b99c3c90, b99c3d24) + c

0x45c448e0: panicSignalCatchHandler + 0x24 (b, b99c3c90, b99c3d24, b, 0, 0) + d4

0x45c0be7e: ExceptionHandlerStub + 0x196 (b, b99c3c90, b99c3d24, 81c8020, b99c3d24, b)

0x45c8412c: userSignalHandler + 0x68 (b, b99c3c90, b99c3d24, 45c0bce8, 81c8020, b99c3d24) + 28

0x45c840ba: intrDispatch + 0xba (b, b99c3c90, b99c3d24, 81c8020, 4206eac0, b99ffc00) + 10

0x45c8431c: intrDispatchMD + 0x60 (b, b99c3c90, b99c3d10, 41f55ca4, 0, 8c9ddd0)

0x42063228: pthread_sighandler_rt + 0x78 (b, b99c3c90, b99c3d10, b, 0, 1) + 364fc

0x420d1d60: __libc_sigaction + 0x120 (8c9ddd0, 94ed52c, 41f55ca4)

0x4157046e: genouterstmt__9SIcParser + 0x42 (8c9ddd0, 41f55ca4, 8c9ddd0, 956a6d0, 0)

0x41570386: WalkCodeGen__9SIcParserP7LSsNode + 0xb6 (8c9ddd0, 956a6d0, 41f55ca4, 4156e5c0, 8c9ddd0)

0x4156e64f: _script__9SIcParser + 0x8f (8c9ddd0, 41f55ca4, 41a2c340, 8c9ddd0, 1a48c)

0x4155f8cd: attach__9SIcParser + 0x1b9 (8c9ddd0, 41f55ca4, 47382f50, 8c9ddd0, 40796c76, 41f55ca4) + 26c

0x4155eebb: Parse__9SIcParser + 0x75b (8c9ddd0, 8c9ddd0, 41f55ca4, 47382f50, 83f98b0, 8dfbe50) + 3e0

0x4155d97d: doParse__9SIcParserP9LSsModulePFP11LSsInstanceP9LSsModuleP11LSMEMSOURCE_UsPFP11LSsInstanceP9LSsModuleUsUsUsPvPv_s + 0x27d (8c9ddd0, 47382f50, 4152a1f0, 415296b8, 41f55ca4, 47382f38) + b0

0x41531b35: ExecuteCompile__11LSsInstancePUsP11LSMEMSOURCEPvPP9LSsModule + 0x199 (83f98b0, 840de98, b99fa998, 47382f38, b99fa994, 41f55ca4) + 28

0x4154081d: ExecStatement__9LSsThread + 0xd1 (92a7940, 41f55ca4, 0, b99faab2, 415c715b, 81f2cc4) + cc

0x415a244d: NRun__9LSsThread + 0x2d69 (92a7940, 41f55ca4, 0, 49ed91e8, 81f2cc4, 1) + ac

0x415a271c: Run__9LSsThread + 0xec (92a7940, 41f55ca4)

0x41523804: Resume__11LSsInstance + 0x30 (83f98b0, 41f55ca4, 0, 49ee78a8)

0x4151eea2: Run__9LSIThreadUli + 0x7a (49ed91e8, 78, 1, 41f55ca4, b99face8, 49ee78a8) + 10

0x4151ef47: RunInternal__9LSIThreadsUlG11tagTIMEDATEi + 0x4b (49ed91e8, 2, 78, 3574c1, c1256fb6, 1) + 38

0x4151f230: RunToCompletion__9LSIThreadUli + 0x1b4 (49ed91e8, 78, 1, 49ed91e8, 49ee85e8, 41f55ca4) + 100

0x41518ee9: RunScript__12CLSIDocumentUisPcsUlUl + 0x399 (49ed0e68, b7, 1, 41a227e0, 0, 78) + 60

0x414af593: Run__21CRawActionLotusScriptP13CDefActionCtxUiPUl + 0x273 (49ee7868, 49ed1ee8, ba, b99fadd4, 41f55ca4, 0) + 10c

0x4149ffe2: Run__10CRawActionP13CDefActionCtxUiPUl + 0x56 (49ee7828, 49ed1ee8, ba, b99faf00, 41f55ca4, b99fb660) + 18

0x4149f87c: Execute__10CRawActionP13CDefActionCtx + 0x110 (49ee7828, 49ed1ee8, 41f55ca4)

0x414a86a3: RunAlone__10CAssistantP13CDefActionCtx + 0x23 (49ef16e8, 49ed1ee8, 41f55ca4, b99fb7c0, b99fb8c0, 3) + 81c

0x414a816d: Run__10CAssistantP21tagASSISTRUNCTXSTRUCT + 0x138d (49ef16e8, b99fb944, 428b37d0, b99fcde8, b99fcde8, 41f55ca4) + 1e8

0x414bfe9a: AgentRun + 0x51e (49ef16e8, 49ed1ee8, 0, 10, 428b37d0, b99fcde8) + b8

0x423d1b56: Run__6NAgent + 0x12e (b99fcde8, b99fcde8, ba, 428b37d0, 4313880c, 49ecbee8) + 1484

0x42478809: HandleOpenAgentCmd__10CmdHandlerP12OpenAgentCmd + 0x295 (4313880c, 49ecbee8, 428b37d0, 49ecbee8, 0, b99fcf88) + ac

0x4234235a: PrivHandle__14CmdHandlerBaseP3CmdT1 + 0x2be (4313880c, 49ecbee8, 0, 428b37d0, 480bbd04, 49ecbee8) + 8

0x4233ea9f: PrivHandle__10CmdHandlerP3Cmd + 0x7f (4313880c, 49ecbee8, 428b37d0, 480bbd04, 0, 4064734e) + 1138

0x4233ecd1: Handler__10CmdHandlerP3CmdPv + 0x1c5 (49ecbee8, 4313880c, 428b37d0, 480bbd04, 0, cb) + a4

0x4232c5cb: Execute__3Cmd + 0x123 (49ecbee8, ffffffff, 7, 428b37d0, 0, 4a039e0c) + 13d8

0x423b3ae1: InotesHTTPProcessRequestImpl__FP18_InotesHTTPrequest + 0x835 (4a039e18, 400957a0, 0)

0x423b3202: InotesHTTPProcessRequest + 0x1a (4a039e18, 400957a0, 0, 4a039d68, 4a039c0c, 4007c00a) + 3f8

0x4004c98b: ProcessRequest__15HTInotesRequest + 0x37 (4a039e0c, 400957a0, 4a039c0c, 0, 3)

0x400476fc: ProcessRequest__21HTRequestExtContainer19HTApplicationSpaces + 0x190 (4a039d68, 3, 400957a0, 49ec8f1c, 430c7fe0, 400957a0) + 8

0x40059619: ProcessRequest__9HTRequest + 0x73d (4a039c0c, 400957a0, 430e4f68, 0, 7d0, b99ffab8) + 8

0x4005f05e: StartRequest__9HTSession + 0x36e (430c7fe0, 400957a0, 430e4f68, 430e4f70, b99ffab0, 400456e2) + 14

0x4006c6eb: CheckForWork__14HTWorkerThread + 0x173 (430e4f68, 400957a0, 430e4f68, 3)

0x4006c506: ThreadMain__14HTWorkerThread + 0x7a (430e4f68, 430e4f68, 430e4f68, 41f55ca4, b99ffc00)

0x400683ad: HTThreadBeginProc + 0x3d (430e4f68, 438d5f74, 4206eac0, b99ffc00, 0, 0) + 8

0x4065c28f: ThreadWrapper + 0xfb (0, 0, b99ffd28, 0, 0, 0) + c0

0x420601b0: pthread_start_thread + 0xf0 (b99ffc00, b99ffc00, 0, 0, 0, 0) + 466003ec

2

Subject: HTTP Server Crash

Hi Jose,

It’s a known problem, this is a tn (1169330)about it (see the Stack trace from AIX) :

Problem

Using the LotusScript Split function as a parameter in a function call results in a crash or hang. This typically occurs in cases where the call to the function is preceded by a call to the Evaluate function.

For example:

variableA = Evaluate(|…|, doc)

…

Call Somefunction(Split(variableX))

Here are some samples of the fatal threads which have been observed:

Stack trace from Win 32 Client

############################################################

FATAL THREAD 1/9 [ NLNOTES:0308: 288]

FP=0x0012e9ac, PC=0x6010abe4, SP=0x0012e9a8, stksize=4

EAX=0x007f00bf, EBX=0x04aa0001, ECX=0x00e35678, EDX=0x04aace10

ESI=0x04aace10, EDI=0x00e35678, CS=0x0000001b, SS=0x00000023

DS=0x00000023, ES=0x00000023, FS=0x00000038, GS=0x00000000 Flags=0x00010202

Exception code: c0000005 (ACCESS_VIOLATION)

############################################################

nnotes.LSsInstance::UnLinkDynArray

nnotes.LSsModule::ClearRTLinks

nnotes.LSsModule::PreDestruct

nnotes.LSsModule::Destruct

nnotes.LSsInstance::DeleteModule

nnotes.LSsInstance::ModuleUnload

nnotesws.LSEcInstance::UnloadScriptModule

nnotesws.LSEcInstance::UnloadUsedModule

nnotesws.LSEcInstance::UnloadDocumentScripts

nnotesws.LSEcSAScrEnv::UnloadDocumentScripts

nnotesws.LSEcSAScrEnv::DoMethod

nnotesws.LSEcSAScrInt::ClassControl

nnotes.LSsInstance::AdtCallBack

nnotes.LScObjCli::ProdMethodCall

nnotes.LSsThread::AdtDoProdCallout

nnotes.LSsThread::AdtCallMethod

nnotes.LSsThread::NRun

nnotes.LSsThread::Run

nnotes.LSsThread::ExecuteEvent

nnotes.LSsInstance::RunEvent

nnotes.LSsInstance::RaiseEvent

nnotesws.LSEcSAScrInt::DoRaiseEvent

nnotesws.LSEcSAScrEnv::DoRaiseEvent

nnotesws.LSEcInstance::DoScrEnvEvent

nnotesws._LSEObjectDeleteAll@8

nnotesws.NotesIM::CleanupDoc

nnotesws.CLSUIDocument::~CLSUIDocument

nnotesws.CLSUIDocument::`scalar deleting destructor’

nnotesws.CUIActionCtx::~CUIActionCtx

nnotesws._RunAssistantEx@28

nnotesws._RunAssistant@20

nnotesws.CAgentDocSelection::Command

nnotesws.CSelectionManager::ProcessCommand

nnotesws._ProcessWMCommand@8

nnotesws._MainWndProc@16

nnotesws._PeerFrameWndProc@16

USER32.MsgWaitForMultipleObjects

USER32.TranslateMessageEx

USER32.DispatchMessageW

NLNOTES._WinMain@16

NLNOTES._WinMainCRTStartup

KERNEL32.OpenEventA

Stack trace from AIX on HTTP task

###################################

thread 31/62 :: http pid=31502, k-id= 137823 , pthr-id=7711

stack :: k-state=wait, stk max-size=262144, cur-size=72252

###################################

ptrgl._ptrgl() at 0xd01d0b38

raise.nsleep(??, ??) at 0xd01def50

raise.nsleep(??, ??) at 0xd01def50

sleep(??) at 0xd0247570

OSRunExternalScript(??) at 0xd2b7f61c

OSFaultCleanup(??, ??, ??) at 0xd2b8019c

fatal_error(??, ??, ??) at 0xd32b59f0

LiteralFree__11LSsInstanceFP15LSsStringHeader(0x37285b20, 0xbf0040) at 0xd31d7790

ArrayErase__9LSsThreadFP8LSsValue13LSeThArrErase(0xd31d7824, 0x37f25328, 0x352f8ef0) at 0xd31f7a14

LsValClean__9LSsThreadFP8LSsValuei(??, ??, ??) at 0xd320644c

PopValueComplex__9LSsThreadFv(??) at 0xd32176dc

NRun__9LSsThreadFv(??) at 0xd3231fe0

Run__9LSsThreadFv(??) at 0xd32345ac

Resume__11LSsInstanceFv(??) at 0xd31b2c90

Run__9LSIThreadFUli(??, ??, ??) at 0xd3280064

RunInternal__9LSIThreadFsUl11tagTIMEDATEi(0xd3289418, 0x0, 0x352f92a0, 0xd327c6a0, 0x352f9308, 0x36ed9ce0) at 0xd32802e8

RunToCompletion__9LSIThreadFUli(??, ??, ??) at 0xd3280688

RunScript__12CLSIDocumentFUssPcT2UlT5(??, ??, ??, ??, ??, ??, ??) at 0xd32850e8

Run__21CRawActionLotusScriptFP13CDefActionCtxUsPUl(??, ??, ??, ??) at 0xd2fe27c8

Run__10CRawActionFP13CDefActionCtxUsPUl(??, ??, ??, ??) at 0xd2fcd9e8

Execute__10CRawActionFP13CDefActionCtx(??, ??) at 0xd2fcd718

RunAlone__10CAssistantFP13CDefActionCtx(??, ??) at 0xd2fd818c

Run__10CAssistantFP21tagASSISTRUNCTXSTRUCT(??, ??) at 0xd2fd7bc8

AgentRun(??, ??, ??, ??) at 0xd2fcaf14

Run__6NAgentFv(??) at 0xd4dc96d0

DoToolsRunMacro__8NFormulaFPUlPPv(??, ??, ??) at 0xd4b9b1b8

ExtensionProc__8NFormulaFUsT1PUlPPvPUsT3(??, ??, ??, ??, ??, ??, ??) at 0xd4b962f4

INotesCompExtProc(??, ??, ??, ??, ??, ??, ??, ??) at 0xd4b91088

ExtensionProc__18CompGeneralContextFR7ComputeUlPP9CompValueT2(??, ??, ??, ??, ??) at 0xd3c168c0

Execute__13ExtensionProcFv(??) at 0xd3c4c34c

ComputeVariants__14AtFunctionNodeFv(??) at 0xd2c3e808

ComputeVariants__18MainExpressionNodeFv(??) at 0xd2c3fe50

ComputeVariants__8RootNodeFv(??) at 0xd2c40264

Eval__7ComputeFv(??) at 0xd2c38900

NSFComputeEvaluateExt(??, ??, ??, ??, ??, ??, ??, ??) at 0xd2c35a18

Evaluate__8NFormulaFR6NValuePi(??, ??, ??) at 0xd4b91e4c

Evaluate__8NFormulaFv(??) at 0xd4b92b10

HandleClick__10CmdHandlerFP3CmdR5NNoteR11ClickTargetR6StrBufP10SelectList(??, ??, ??, ??, ??, ??) at 0xd4c9a4a8

HandleEditDocumentCmd__14CmdHandlerBaseFP15EditDocumentCmd(??, ??) at 0xd4e7d730

PrivHandle__14CmdHandlerBaseFP3CmdT1(??, ??, ??) at 0xd4c1c778

PrivHandle__10CmdHandlerFP3Cmd(??, ??) at 0xd4c20d48

Handler__10CmdHandlerFP3CmdPv(??, ??) at 0xd4c2107c

Execute__3CmdFv(??) at 0xd4adbcbc

InotesHTTPProcessRequestImpl__FP18_InotesHTTPrequest(??) at 0xd4c30a38

InotesHTTPProcessRequest(??) at 0xd4c301bc

ProcessRequest__15HTInotesRequestFv(??) at 0xd49a70e4

ProcessRequest__21HTRequestExtContainerF19HTApplicationSpaces(??, ??) at 0xd49cd848

ProcessRequest__9HTRequestFv(??) at 0xd49d6f50

StartRequest__9HTSessionFv(??) at 0xd49e1058

CheckForWork__14HTWorkerThreadFv(??) at 0xd49e8750

ThreadMain__14HTWorkerThreadFv(??) at 0xd49e8390

HTThreadBeginProc(??) at 0xd49b1dac

ThreadWrapper(??) at 0xd29b3b94

pth_pthread._pthread_body(??) at 0xd017156c

Solution

This issue has been reported to Lotus software Quality Engineering.

Workaround:

Assign the result from Split() function to a varaible before calling the user defined function. For example:

variableA = Evaluate(|…|, doc)

…

variableY = Split(variableX)

Call Somefunction(variableY)

3

Subject: RE: HTTP Server Crash

We’ve probed that and it works (not hangs the server). Do you know anything else?.

Thanks in advance!!