View index failure

  • My application logs to a different database, and that database is FT Indexed to speed troubleshooting. Since migration to R8.5.1, the FT search consistently fails to find documents that I know for an absolute fact do match the criteria. For instance, just now I searched for 4a01e240800e25bf, a control number relating to the random save conflicts I’m seeing. The document normal path is: Import → Users process. That means nothing what-so-ever should exist without an import log. Yet when I search for the above control number, I see only user interaction logs, and not even all of them. If I go to the import logs, open one that should contain the above control number, and do a “Find” on it, I see the import log does in fact contain an entry for 4a01e240800e25bf. But FT Search never finds that log, nor any other import log for this control number.- It’s not a stale index. Three days ago I deleted the FT Index and gave it two days without an index, then yesterday I recreated the index from scratch. It said 1 document is not indexed (normal, as the users are adding logs all the time). No dice. Then I deleted the index again, ran fixup, and rebuilt the index from scratch. That brings me to today, where 4a01e240800e25bf still returns no import log hits.

  • I see nothing under “More” for search parameters that would exclude anything, and I’m doing a simple text search, so I’m not excluding any potential matches. If I add a search term to include only import logs, it returns zero hits. It’s as if nothing but user interaction logs actually exist in the index.

  • If I tell it to find nothing but import logs, it returns exactly 3 hits, all of which are connection timeouts on the remote system interface. There are hundreds if not thousands of import logs.

  • These logs are all the exact same Form with the exact same access control, and I can obviously access the logs because I may open and search them manually. What makes an “import” or “user” log is a field used for categorization in the only View. There is no discernable reason why it would fail to find these.

  • Under R6.5 this was never an issue. Stale indexes were, but a freshly indexed database never failed to return an expected quantity and variety of hits.

  • This is a serious issue, not because it hampers troubleshooting, but because the users rely heavily on FT search in the application. If it’s not showing them all the hits, as the log is not showing them all to me, and data are not processed, the feces will ultimately impact the rotary oscillator.

  • As a side note, the above control number does in fact work as expected in the production application. But I need faith it’s working 100%, and I don’t have that at present.

  • Any ideas?

Thanks for your time…

Subject: Odd. I can’t edit my own post … this is solved…

  • FTG_INDEX_LIMIT does work on windoze, and fact someone (else) had set this to 160000. Since all import logs are over 160,000 Bytes, not a one of them got indexed. All’s well that ends well.

Hope this saves someone some headache…

Subject: Hmm…

We have some very large, heavily used, FT indexes here. Though we’re not indexing attachments.

Are you running this FT search from a view, or from a NotesDatabase.FTSearch() call?

If you haven’t tried the latter then I’d go that route, as your doc may simply not be appearing (or appearing properly, e.g. response hierarchy and stuff) in the view.

If the backend FTSearch call doesn’t find it then I’d open a support PMR.

Subject: I haven’t tried a code-based search…

  • I’m not convinced that’s the issue, however, as I am still continually seeing configured max index size exceeded on the log database. Until that is resolved I have no reason to expect the missing hits are being included in the index at all.- I intended to try the “AS400 only” notes.ini setting last night, just to see what it did, but got pulled away before that was done. I’m going to try that tonight, and if it doesn’t work I’m going to open a PMR on the index size error.

  • The truly odd thing is this has been working for years, on this very log database, with a similar tens of thousands of entries. Why this “exceeded” error all of a sudden? I also intended to bounce the server last night to see if that didn’t just “fix” it … on windoze that is often a simple “solution”.

  • The production application appears to be indexing normally, and it appears to be doing searches normally. As long as that appearance is accurate, this issue with the log database is not nearly so serious.

Thanks for the suggestion!..