Problem with 'Ghost' Response Documents

HCL Domino Domino Forum
1

In an application we sometimes have a problem with some response documents created by Lotus Notes.We call these documents “Ghost”-Documents.

These documents have no fields and no form and cannot be displayed in views.

So, if we want to access a response document collection with Lotusscript to get some field values from the response document we always get an error, if the response is a “Ghost”-Document.

Also it is not possible to delete such a response because of missing fields and rights.

Does anybody know this problem?

What can cause this problem?

What can be done to remove those documents from the database?

Any hints are appreciately welcome.

2

Subject: Deleted docs?

Once you have a handle to a “ghost” doc (via LotusScript) try looking for the “IsDeleted” property. I’ve occasionally had deletion stubs come back into a doc collection (either responses or via a search). If they are deletion stubs and you don’t have replica’s anywhere else, you can try purging them by setting the “Remove documents not modified in the last x days” option on Space Savers under Replication options. Set it to 3 or less days and DON’T enable the property. Deletion stubs are (or used to be) removed in 1/3 the time displayed on that option.

3

Subject: my theory

(I originally posted this in the R7 forum, but I thought I’d re-post it here to see if anyone else has had similar experiences)

Does anybody know this problem?

Yes I believe I’ve seen this. It sounds to me like an orphan document (a response with no parent) was replicated to another database where it previously didn’t exist. From what we have seen in our experiments, it looks like Notes will not tolerate a new response being created with no parent, so it will create a new empty document with the orphan’s $ref as its UNID.

What can cause this problem?

Perhaps the deletion stubs expired for the deleted parents of the response documents before you replicated?

What can be done to remove those documents from the database?

That depends a bit on the circumstances. If the orphans are worth keeping, you could just delete the $Ref item in them (scanEZ will let you do this in a mass-modification operation). Another approach might be re-assigning these orphans to new parents. Or if possible, you could just delete all the troublesome orphan documents.

Full disclosure: I work for Ytria as a tech writer.

Best regards,

Peter

4

Subject: Further Information

Please note that there is one very strange think:The ghost documents does not have an item $Ref but Notes discovers them as response documents anyway.

It seems that the ghost documents does not have any items, so it is also not possible to display those documents within views.

5

Subject: my theory part II

Hi again Christian,

Your situation sounds exactly like the one we’ve seen.

Just to clarify, the “ghosts/poltergeists/ancestral specters” have no items at all–I just meant to say their UNID matches the $Ref field of an orphan document.

Here’s the logic of the procedure we used to find them:

Find the responses (i.e. the documents which contain a $REF item) then get their parent documents (from the UNID in the $REF). Now see if the size of any of these parents = zero. If the parent has no size, chances are it’s a ghost/poltergeist/ancestral specter/whatever-you-want-to-call-it.

We used scanEZ to do this (naturally!) but I can’t see why it wouldn’t be possible to accomplish this using LotusScript.

Could you clarify the bit about compacting databases? Is that creating the problem docs or removing them?

Regards,

Peter

6

Subject: Some remaining questions and a possible solution

Hi again Peter,

I have read your information and it sounds good but some questions still remain. Maybe you can answer them.

I just meant to say their UNID matches the $Ref field of an orphan document.

But isn´t it the other way round? The ghost document is a child of a normal document (which is remarkable as it does not even have a $REF!).

why it wouldn’t be possible to accomplish this using LotusScript.

It´s possible, but it´s difficult and not without risk. You cannot delete the doc itself. You can see the doc in the debugger, but in LS you don´t get a handle. A doc.remove command will result in an “object variable not set” error (like any other command which tries to do something with this document). Actually, there is a way to kill the doc: The ghost document appears only in document collections, never “alone”. So when you remove the entire collection in which it is contained, then the ghost will be removed, too.

So, this would work:

  1. Get a collection of the parent´s responses (doc.responses)

  2. Loop through these responses: Put each “good” document in another collection and remove its child status. Important: you cannot loop through this collection via col.getNextDocument (doc). This will lead to the “object variable not set” error described above. You have to use the col.getNthDocument (i) with an appropriate error handling.

  3. Get again a collection of the parent´s responses

  4. Remove this collection

  5. Get the documents you stored away in step 2 and reattach them to their former parent.

This works, after this procedure the ghost document is gone. But that´s just not a viable way for us. It would be acceptable as a sort of repair tool. But the problem happens far too often. And we cannot put such a procedure in the regular code, as it would inflate the replication.

Best regards

Christian

7

Subject: A change of course…

Hi again Christian,

Sorry, I was a bit mixed up. I think this issue of yours is a quite a bit different from the one that I had been talking about.

In my case the ‘poltergeists’ were definitely parents. In your case, (after re-reading your information) I suspect there may be an inconsistency between the response table and the actual contents of the database itself. In fact, some of your documents have incorrect response table. The response table is a list that helps to let a document know its responses.

What truly decides whether a document is a response or not is the presence of a $REF item. The response list in the parent is just here to help finds the responses of a doc without having to parse the full database.

However, this table can be altered but yet still contain NoteIDs for documents that no longer exist or that have became the response of another doc. If I remember correctly, this is especially true of R5…

So in your case, you’ll perceive that there is a document here (because the reponses property lists it in Lotusscript), but in reality it points to nothing: the document does not exist.

We encountered this situation in some of our databases.

I unfortunately don’t know any way to fix this situation (we tried a lot of different things). As you say, sometime a compact does the trick, sometime it doesn’t.

In Lotusscript, what we did to protect ourselves against this was to do the following every time we parse the list of responses:

Set ndcResp = nd.Responses

If ndcResp.Count > 0 Then

Set ndResp  = ndcresp.GetfirstDocument 

Do While Not ndResp Is Nothing 

	If ndresp.isvalid And Not (ndresp.isdeleted) Then ' bcose some responses can stay in the list but are deleted 

		If ndresp.ParentDocumentUnid = nd.UniversalID Then ' bcose responses table is not always accurate 

			' Then you can do your stuff here: the response is a valid one... 

		End If 

	End If 

	Set ndResp = ndcResp.GetNextDocument(ndResp) 

Loop

End If

If you download the free version of scanEZ ( Getting Started with scanEZ ); open the database in question; click on one of the parent documents in the selection tree, you’ll be able to see any ‘phantom responses’ listed in the parent/response tab. You will see that on a given document with problems, this parent/response tab gives you (for example) 3 responses, but in the list you will see some of the responses are actually deleted or non-existent (a big error string will be given, instead of the list of the items for the response document)…

In the case of an empty parent, the document exists (you can open it using scanEZ or anything else), but is completely empty (not a single item inside).

Have a great weekend,

Peter

8

Subject: Compact to remove ‘Poltergeists’

Hello again Peter,

normaly compacting a database with some advanced options is a good way to remove “Poltergeists”.

We use the following options (entered directly at the server console):

load fixup [Database] -F

load compact [Database] -c -D -i

load updall [Database] -R -X

Unfortunately this does not work always.

9

Subject: Ghosting response

I hope this helps – in the “Administrative considerations” topic in the Alloy admin help (at http://publib.boulder.ibm.com/infocenter/domhelp/v8r0/topic/com.ibm.help.atlantic.admin.doc/atl_consids_c.html) we say this about ghosting:

To enable “ghosting” in a calendar view, do the following in Notes:

Note: You do not need to change any scheduling preferences; do this once to permanently activate ghosting.

Click Open > Calendar.

Click Tools > Preferences.

Click the Calendar &ToDo and Scheduling tab, and then click OK.

Note:

See “Enabling calendar ghosting in IBM Lotus Notes V8” at http://www.ibm.com/developerworks/lotus/library/notes8-ghosting/index.html?S_TACT=105AGX13&S_CMP;=EDU.

10

Subject: This is something different

Thank you for your response but I’m sorry to say that this is something completely different.

In my case we do nothing with the calendar.