In my database i have 1000’s of documents. Most of them appear just fine, but a few hundred of them won’t appear in any views. I believe the problem is due to too much data in the documents.
I used to get errors about exceeding the 32k limit on Summary items when I attempted to save the documents. I trimmed down the documents and now I am able to save.
However, the records disappear from all the views.
If I do a db.search with |Form = “Main”| then I can get access to them, but that is the only way.
The summary items I am using are actually READER fields, so they have to have the isSummary flag. I don’t need to display this value in any columns, so I don’t understand why the doc disappears.
Subject: RE: Documents with lots of data not appearing in views
Hi Yannick,
I just finished dealing with this issue for the Stats Dept of NZ. A few pointers on isSummary:
1 - If the field that is set to isSummary = false is in the view coloumn formula it will not appear in the view
2 - If the field is in the view selection formula it will not appear.
I have a bunch of agents and functions now for dealing with the isSummary problem. We spent months going ape trying to sort out a Db with 800 ooo docs in it. I figureI might now be the world expert on this subject
Subject: RE: Documents with lots of data not appearing in views
Agreed, and another point about large ReaderNames fields …
You should always try to remove individual names and replace them with [roles]. Ditto for groups.
It certainly is possible to have Reader Names fields which are not SUMMARY, but of course they’ll lose their reader powers at the view level. Users still won’t be able to perform doc properties or open the document, but they’ll see the document in the view. I’m not saying this is a good thing, but for certain applications it actually works just fine. Just think through your requirements.
Subject: RE: Documents with lots of data not appearing in views
I am not trying to do anything special with the documents. The summary items that are too large are not being displayed in the views or used for the selection formula.
All they are for is to store Reader names. But the isSummary flag has to be turned on for the Reader access to take affect. I can’t turn these into RT fields either.
Say I start with an item that is roughly 32k. I am splitting that item up into smaller ones, such as:
Security1, Security2, Security3, etc.
Each individual item may be 8k or smaller.
But it doesn’t matter, the documents will not appear in the views.
I believe its a problem with the Indexer, and the overall total of all summary items being greater than 32k.
Any ideas on database properties or anything else I can check? How about server settings?
Using the ScanEZ tool, I can see that my missing documents are considered to be Deletion Stubs.
Any idea why this is?
If I strip out my large summary fields, then the document reappears in the view and it is no longer considered a deletion stub. But i lose my READER fields of course.
What can I do to ensure that it is not marked as a deletion stub in the first place.
65k is the correct limit for the sum of all summary fields. I’ve found experimentally that I usually am safe up to mid 50’s, and then I start running into trouble. I assume this is because there is overhead in many ways which we are not accounting for. Why you would run into problems at 43k? Maybe you’re not counting carefully enough, or maybe the overhead varies radically depending upon factors we’re not considering, such as number of fields.
There is no way around this limitation of summary data. You will need to either make some summary fields non-summary, or you will need to make some summary fields smaller.
I’m not familiar with the tool you used, but your other tests have confirmed that the documents are not really deleted, so that’s that. They don’t appear in views because of your summary data limit problem, not because they’re deleted documents.
"These errors occur when a particular Text field in the document or the document itself has reached the maximum amount of Summary data that it can hold.
Notes/Domino 6.x and 7.x have the following limits regarding Summary fields: 32k per field, 64k per document. "
However, in one of my later posts, I mentioned how my doc totals are only 43,000 bytes. I ran a lot of tests and it seems like 42k is acceptable, but anything over that is too much.
What is making these things act like Deletion stubs???
