HORRIBLE SPR - Please bring your Lotus Support contracts to bear on this one

I wouldn’t normally poll the forum like this, but there is a significant regression bug in ND6/7/8 that still exists in 7.0.3, 8.0.1 and 8.5 Beta 1. The ticket on this SPR (# DCOE6N82TP) has been open a couple of years because it is such a horrific bear to reproduce.

IBM has finally acknowledged that it is a bug, and a fairly significant one. However the fix for it will require an ODS update, meaning that 8.5.1 is the first candidate release for a fix. It needs just a bit more weight added to it though before they’ll commit to the fix, so if you find this bug important please open a PMR ticket and ask to have SPR # DCOE6N82TP fixed.

The Short Story:

Documents can – on rare occasion – remain in a view even though they should no longer be there according to the view’s selection formula.

This problem affects any Domino database with the database property “optimize document table map” enabled. Which happens to be many core Domino databases, including the NAB.

Refreshing an affected view (e.g. F9 in the client) does NOT fix the problem.

Rebuilding the view index via one of the following methods will fix the problem, temporarily:

  1. Shift-F9 in the client

  2. A Fixup followed by Updall -R on the affected database

  3. Ctrl-Shift-F9 in the client

Once fixed, the problem may never manifest in that DB again. Or it may re-manifest in the same view (or a different view) in 5 minutes. It’s very intermittent.

The problem is also extremely rare – we do massive amounts of doc updates daily and are only seeing this problem manifest a couple of times a week. It happens without warning – potentially after years of trouble-free view indexes.

Turning off the database property (followed by a compact) will avoid the problem, but at a massive performance penalty (see below).

Why should you care?

  1. This is a cornerstone of Domino workflow functionality:
  • doc appears in a view

  • doc’s fields get changed

  • doc should no longer appear in view <<< with this bug, this step is not 100% reliable

  1. This property is enabled by default on many core Domino database (including the NAB). Due to the extreme rarity and intermittence of the bug, it is possible that a wide variety of all-around “strange” behavior is related to this obscure problem.

  2. If you have any custom apps on Domino, this bug can obviously cause severe problems.

  3. Since fixing this bug will require an ODS update, it must get fixed in a major release. This bug has taken literally years to track down so if a plan isn’t made to address it in early 8.5, it will need to wait until N/D 9 or later.

“I’ve seen this property but I don’t know what it does exactly…”

From the Help:

That’s a bit vague, though. Here’s an example that might make things more clear. Picture a database with 100,000 docs:

99,500 form field of “A”

 500 form field of "B"

With “Optimize Document Table Map” OFF, A view or NotesDatabase.Search() with a formula of SELECT FORM=“B” will iterate through ALL 100,000 docs looking for matches.

WIth “Optimize Document Table Map” ON, A view or NotesDatabase.Search() with a formula of SELECT FORM=“B” will iterate through ONLY the 500 “B” docs looking for matches. The other 99,500 docs are skipped.

This can result in HUGE performance improvements for large DBs with small subsets of documents. This property worked brilliantly in R5, but has been glitchy since ND 6.0.

Thanks for reading. Without this bug being fixed, we all have two options:

  1. Accept “mostly accurate” view indexes throughout many DBs in Domino (including the NAB, which will cause general strangeness on occasion).

  2. Avoid using one of the most performance-enhancing options of Domino.

Subject: HORRIBLE SPR - Please bring your Lotus Support contracts to bear on this one.

This post has nothing to do with your request, which I favour, but I wonder how you were able to enter bold and other font settings in a post. I tried to use and tags in my posts, but that does not work.

Subject: RE: HORRIBLE SPR - Please bring your Lotus Support contracts to bear on this one.

You need to use the Notes client to post. There’s a link on the right that will give you info on how to get native Notes access.

Subject: RE: HORRIBLE SPR - Please bring your Lotus Support contracts to bear on this one.

Stan, thanks for the tip. It works!

Subject: RE: HORRIBLE SPR - Please bring your Lotus Support contracts to bear on this one.

Welcome to the club! I find the client to be much better for browsing/posting in the forum.

Subject: RE: HORRIBLE SPR - Please bring your Lotus Support contracts to bear on this one.

You could be right by stating that the client is much better for browsing/posting.

When I close the browser I cleanup all private stuff, which means that I loose my read-marks. And that is not always nice. I will give it a try to see what I like most.

Subject: HORRIBLE SPR - Please bring your Lotus Support contracts to bear on this one.

Erik,

There is internal discussion to determine the proper course of action for this SPR. Please note that this is a very rare issue that appears to be limited to documents whose form has changed. We do not currently believe that this behavior would be seen in databases that do not have this type of workflow, with the NAB being an important example.

For now, the recommendation is to disable the “Optimize document table map” flag for any database that tracks a note’s form changes for workflow.

Subject: Re: Response

Thanks for the response, Chad. We are definitely aware of the rarity of the problem – for us, we’ve chosen to deal with the problems rather than disable the db property. The impact of the performance hit is simply too great and would result in massive scalability problems.

The major concern with this problem is, in fact, its rarity: it is nearly impossible to reproduce on-demand. With such rarity I would venture that it’s very likely there are others experiencing this problem that don’t even know it, and probably other PMRs opened in the past that are influenced by this problem.

Our PMR was finally listed as “closed, no plans to fix without additional weight”. If that’s not the case then we were given misinformation, and it’s possible I may have jumped the gun with my post. But going off of the knowledge available hopefully you’ll understand our attempt to add more weight to the SPR.