db.UnprocessedDocuments -- Is There Any Kind of Predictability?

Hi,

I know that using the NotesDatabase UnprocessedDocuments property on documents selected in a view results in documents being processed in a seemingly random fashion – not in the order in which they appear in the view. [This is a HUGE annoyance, by the way!] This is not documented in Designer Help, but I was wondering if anyone knew if the documents returned are at least grouped under the same category, if it is a categorized view?

I have to add up hours in timesheets of different people and produce an invoice. The view is categorized by project, then again by employee name. I have found during preliminary testing that Notes appears to at least process all the timesheets of one person before moving on to another, but I wonder if I can rely on this to always be the case. [I am already dealing with the hassle of having to reorder the timesheets I display in the invoice because they jump around date-wise since UnprocessedDocuments does not process them in order, which is sorted by weekending date!]

Thanks in Advance,

Ken

///// Update: I found that the answer to this question is no. The docs are totally random and not grouped by category or anything else. I used the technique from Eshter’s 2nd code sample to put the collection into a folder and then process the docs in order as a view. /////

Subject: db.UnprocessedDocuments – Is There Any Kind of Predictability?

It’s not that UnprocessedDocuments is random, but that NotesDocumentCollections are never sorted unless they’re the result of a search. (“Selection in a view is not a search,” he said sarcastically.)

There are a couple of ways to order the documents. One is to create a folder that is sorted/categorized the same way as the view, and put the collection into the folder. You can then walk the folder. You can also create an empty document collection and put the documents into the new collection by comparing documents in the old collection against the view.

A third method of attack, and one that makes more sense to me in the scenario that you describe, is to forget about UnprocessedDocuments altogether. The choice to include/exclude timesheets for invoicing shouldn’t be a run-time decision made by the user – there’s got to be some programmatic way of determining what to include. Once you figure that out, it’s a simple matter of walking a view and grabbing documents as appropriate.

Subject: db.UnprocessedDocuments – Is There Any Kind of Predictability?

It’s not really the UnprocessedDocuments that’s the problem - I’m assuming you’re using NotesDocumentCollection = db.UnprocessedDocuments. It’s the NotesDocumentCollection that messes with the order of things. If you look at the Help for the NotesDocumentCollection.IsSorted property, you’ll see the collections are only processed in sort order when they are the result of a full-text search - not when they are created from the UnProcessedDocuments.

That said, there are a bunch of script libraries out there you can download to create a sorted document collection.

Here’s one - can’t remember where I found it. An example of the call to it is below, so read all the way down.

’ ---- This extracts the formulas etc for columns

Class SortedColInfo

Public colType As String

Public formula As String

Public fieldName As String



Sub new(viewCol As notesviewColumn)

	If viewCol.formula = "" Then

		Me.colType = "Field"

		Me.fieldName = viewCol.itemName

	Else

		Me.colType = "Formula"

		Me.formula = viewCol.formula

	End If

End Sub

End Class

’ ---- This holds details of each member

Class SDocColMember

Public SortKey As String

Public Doc As notesdocument



Sub new(doc As notesdocument, sortedColInfo() As sortedColInfo)

	Dim formulaResult As Variant

	Dim tmpKey As String

	

	Set Me.Doc = doc

	

      ' ---- Calculate the sortkey

	tmpKey = ""

	Forall x In sortedColInfo()

		If x.colType = "Field" Then

			tmpKey = tmpKey & doc.getItemvalue(x.fieldName)(0)

		Else

			formulaResult = Evaluate(x.formula,doc)

			If Isarray(formulaResult) Then

				tmpKey = tmpKey & formulaResult(0)

			Else

				tmpKey = tmpKey & formulaResult

			End If

			

		End If

		

	End Forall

	

	Me.SortKey = tmpKey

	

End Sub

End Class

Class SDocCollection

Public count As Integer

Private docCol As notesdocumentcollection

Private parentView As notesview

Private members() As SDocColMember

Private currentPosition As Integer



 ' ---- Sorts the members using the calculated key (stolen from Viviens sort code)

Private Sub SortMembers

	Dim k As Integer, i As Integer, j As Integer, h As Integer, r As Integer

	Dim NumItem As Integer

	Dim tempMember As SDocColMember

	

	k = Me.count

	

      ' ---- Set up for Shell sort algorithm

	h = 1

	Do While h < k

		h = (h*3)+1

	Loop

	h = (h-1)/3

	If h > 3 Then

		h = (h-1)/3

	End If

	

      ' ---- Shell sort algorithm

	Do While h > 0

		For i = 1+h To k

			Set tempMember = Me.members(i-1)

			j = i-h

			Do While j >0

				If Me.members(j-1).SortKey > tempMember.SortKey Then

					Set Me.members(j+h-1) = Me.members(j-1)

					Set Me.members(j-1) = tempMember

				Else

					Exit Do

				End If

				j = j-h

			Loop

		Next i

		h = (h-1)/3

	Loop

End Sub



Sub new(docCol As notesdocumentcollection, view As notesview)

	Dim sortedCols() As SortedColInfo

	Dim viewCol As notesviewcolumn

	Dim doc As notesdocument

	Dim i As Integer, k As Integer

	

	Set Me.docCol = docCol

	Me.count = docCol.count

	Set Me.ParentView = view

	Redim Me.members(docCol.count - 1)

	

      ' ---- Determine which columns in the view are sorted 

	i = 0

	Forall col In view.columns

		Set viewCol = col

		If viewCol.issorted Then

			Redim Preserve sortedCols(i)

			Set sortedCols(i) = New SortedColInfo(viewCol)

			i = i + 1

		End If

	End Forall

	

      ' ---- Step through all of the docments and calculate the sort key etc

	k = 0

	Set doc = docCol.getfirstdocument

	While Not (doc Is Nothing)

		Set Me.Members(k) = New SDocColMember(doc,sortedCols)

		k = k + 1

		Set doc = docCol.getnextdocument(doc)

	Wend

	

      ' ---- Sort the members of this object

	Call SortMembers

	

End Sub



 ' ---- Returns the first document in the sorted collection

Function GetFirstDocument As notesdocument

	Me.currentPosition = 0

	Set GetFirstDocument = Me.Members(Me.CurrentPosition).doc

End Function



 ' ---- Returns the next document in the sorted collection

Function GetNextDocument As notesdocument

	If Me.CurrentPosition + 1 > Me.count - 1 Then

		Set GetNextDocument = Nothing

	Else

		Me.currentPosition = Me.currentPosition + 1

		Set GetNextDocument = Me.Members(Me.CurrentPosition).doc

	End If

End Function



 ' ---- Return the Nth document

Function GetNthDocument(position As Integer) As notesdocument

	If position > Me.count Or position < 1 Then

		Set GetNthDocument = Nothing

	Else

		Me.currentPosition = position - 1

		Set GetNthDocument = Me.Members(position - 1).doc

	End If

End Function



 ' ---- Returns the last document in the sorted collection

Function GetLastDocument As notesdocument

	Me.currentPosition = Me.count - 1

	Set GetLastDocument = Me.Members(Me.CurrentPosition).doc

End Function



 ' ---- Returns the previous document in the sorted collection

Function GetPreviousDocument As notesdocument

	If Me.CurrentPosition = 0 Then

		Set GetPreviousDocument = Nothing

	Else

		Me.currentPosition = Me.currentPosition - 1

		Set GetPreviousDocument = Me.Members(Me.CurrentPosition).doc

	End If

End Function

End Class

To use SortedDocCollection:

'remember to set Use SortedDocCollection in Options

Sub Click(Source As Button)

Dim session As New notessession

Dim currDb As notesdatabase

Dim docCol As notesdocumentcollection

Dim viewName As notesview

Dim doc As notesdocument

Dim sortedCollection As SDocCollection

Dim count As Integer



Set currDb = session.currentdatabase

Set docCol = currDb.UnprocessedDocuments

    	Set viewName = currDb.getview("myView")

Set sortedCollection = New SDocCollection(docCol,viewName)



Set doc = sortedCollection.GetFirstDocument

While Not (doc Is Nothing)

	count = count + 1

	Msgbox "The subject for document number " & Cstr(count) & " is " & doc.subject(0)

	Set doc = sortedCollection.GetNextDocument

Wend

End Sub

Subject: Thanks, Esther!

Esther,

I can’t say I understand that example you posted (what is “Me”? it is not defined), I’ll have to take a closer look. I appreciate it though.

Thanks,

Ken

Subject: RE: Thanks, Esther!

Me is a reserved keyword, used in many programming languages. It doesn’t have to be defined. Take a look at the Designer Help for Class Statement:

"Rules for referring to class members:

You can use the keyword Me to refer to the object itself when you are inside a member procedure. In the example, Me.textColor refers to the value currently assigned to the textColor member of this instance of the class."

But the point is, you don’t have to understand the clas in the script library, just call the function as defined in the sample at the end.

Subject: Didn’t work

Hi Esther,

I never knew it was possible to create classes in LotusScript, I’ve only worked with subroutines and functions, so when I tried out your code I didn’t really know if I was supposed to break it into parts or what. I was surprised that when I copied and pasted it into a new script library under (Options), everything jumped to (Declarations) and stayed there.

Anyway, unfortunately, that code did not work for me. The so-called sorted document collection was no better than the original collection. I don’t know how that code works, but maybe there is something about my view that is throwing it off.

It was worth a try. Next, I will try Stan and Lawrence’s suggestion to use a hidden folder.

Thanks anyway,

Ken

Subject: RE: Didn’t work

No problem; it works for us, but maybe there’s some configuration issue that I missed telling you about.

Yes, you can create classes in LotusScript; there are whole books out there about it. Here’s a good article online if you’re interested:

http://dev.kanngard.net/Permalinks/ID_20020425224026.html

I’ve also posted my full code for using folders in another post - save yourself some work, maybe.

Subject: Another variant

Well … too many lines …My variant (put in view action):

Dim s As New NotesSession

Dim ws As New NotesUIWorkspace

Dim db As NotesDatabase

Dim vi As NotesView

Dim col As NotesDocumentCollection

Dim scol As NotesDocumentCollection 'Sorted by current view collection

Dim doc As NotesDocument

Dim unids As Variant

Set col = s.CurrentDatabase.UnprocessedDocuments

If col.Count = 0 Then Goto ExitSub

Set db = s.CurrentDatabase

Set scol = db.GetProfileDocCollection({SOME_FAKE_KEY})

unids = col.GetNoteIds 'array of noteids (undocumented)

Set vi = ws.CurrentView.View

Set doc = vi.GetFirstDocument

Do While Not doc Is Nothing

If Not Isnull(Arraygetindex(unids, doc.NoteID)) Then

	Call scol.AddDocument(doc)

	If scol.Count = col.Count Then Exit Do 'All found

End If

Set doc = vi.GetNextDocument(doc)

Loop

Subject: Thanks, Stan! (And a question about folders)

Stan,

You always have a lot on the ball, I was hoping you’d see my post.

I read Designer Help and I know it claims that document collections are sorted when the result of a full text search, but the documents are sorted by relevance. How useful is that to a back-end program? At least, I can’t think of a benefit.

Your second suggestion, to do my own sort, sounds daunting. Your third suggestion to scrap the whole UnprocessedDocuments idea and go with a traversing of all documents in a view won’t work at this time – the customer has no clear methodology as to when and how they want to build invoices, so they requested the ability to just select timesheets from a view.

Your first suggestion, to create a folder and put the document collection into it sounds intriguing. I don’t have much familiarity with folders. Can they be hidden to users like views? I guess I am thinking that I wouldn’t want the view visible (maybe even to me!) on an everyday basis. Can I build the folder on the fly, identify what view to model itself after, and delete it afterwards?

Ken

Subject: RE: Thanks, Stan! (And a question about folders)

Yes, you can create a folder on the fly - if you call doc.PutInFolder(“myFolder”) and myFolder doesn’t exist, it will be created.

I don’t know why I didn’t remember this one - I actually used this solution at one point when I couldn’t find the code for the sortedDocCollection. Here’s my code - it takes selected docs, puts them in a new folder created based on the username, and creates an email document that has info from the docs in it, in the correct order.

Sub Click(Source As Button)

On Error Goto errhand

Dim db As notesdatabase

Dim mailDB As New NotesDatabase( "", "" )

Dim doc As notesdocument

Dim mailDoc As notesdocument

Dim mailUI As notesuidocument

Dim dc As notesdocumentcollection

Dim w As New notesuiworkspace

Dim s As New notessession

Dim body As notesrichtextitem

Dim view As NotesView



Set db = s.currentdatabase

Set dc = db.unprocesseddocuments

If dc.count < 1 Then

	Msgbox "You must select at least one document."

	Exit Sub

End If



folderName = "tmpFolder" & s.commonusername

Call dc.PutAllInFolder(folderName)

Set view = db.GetView( folderName )



tempStr = ""



Set doc = view.getfirstdocument





While Not doc Is Nothing

	

	tempStr = tempStr & "Title Code: " & doc.titleCode(0) & Chr(13)

	tempStr = tempStr & "Title: " & doc.title(0) & Chr(13)

	tempStr = tempStr & "PO #: " & doc.poNumberMaster(0) & Chr(13)

	tempStr = tempStr & "BBD: " & doc.bbdActive(0) & Chr(13)

	tempStr = tempStr & "Qty: " & doc.qtyActive(0) & Chr(13) 

	

	tempNeeded  = doc.corNeeded(0)

	If tempNeeded = "" Then

		tempNeeded = "N/A = correction info to come"

	Else

		tempNeeded = Lcase(tempNeeded)

	End If

	

	tempStr = tempStr & "Corrections Needed: " & tempNeeded & Chr(13) & Chr(13)

	

	Set doc = view.GetNextDocument(doc)

Wend



Call mailDB.OpenMail

Set mailDoc = mailDB.createdocument

mailDoc.Form="memo"

Set body = New NotesRichTextItem( mailDoc, "Body" )

Call body.AppendText("This email contains a total of " & dc.Count & " reprint(s)." & Chr(13) & Chr(13))

Call body.AppendText(tempStr)

Call mailDoc.Save(True, True)

Set mailUI = w.EditDocument(True, mailDoc)

Call dc.RemoveAllFromFolder(folderName)



Exit Sub

errhand:

Msgbox  "error # " & Str(Err) & " at line " & Erl & ":  " &  Error

Exit Sub

End Sub

Subject: Haven’t had a chance to try it yet, but thanks!

Esther,

I just wanted to thank you for you last post. I had planned to report back with my results, but sure enough, I’ve spent the last few days fixing problems with other code.

Your code sample for putting the collection into a folder and then using that looks pretty straightforward, and I look forward to trying it out when I can finally revisit this project in the next few days.

Thanks,

Ken

Subject: Esther, it worked great! Thanks so much!

Esther,

I adapted your code sample for putting the random, unpredictable document collection into a folder and then processing the docs in order like a view. It worked like a charm!

I found that the folder still sticks around afterwards, and since I couldn’t find any method to delete the folder after use, other than manually from Designer, I just went into Designer and put parentheses around the folder name, thus hiding it. That will be an acceptable workaround since in my case the same folder name can be reused over and over.

Thanks again!

Ken

Subject: Folders

Folders are exactly like views except that there is no selection formulas. So you want to (a) (one time) create a hidden folder whose columns are sorted in the order you want; (b) put the selected documents in the folder, (c) calculate the invoice by looping through the documents in the folder; (d) remove all the documents from the folder.

Subject: RE: Folders

Lawrence,

Thanks for the explanation!

Ken