Multiple documentCollections : Performance Issues & Most Efficient Way to do it?

HCL Domino Domino Forum
1

I have a database with approx 500,000 documents of which 200,000 I need to create documentCollections from in one regularly used piece of functionality.

For the functionality I need to create 3 collections:

1> Docs created in last 6 months;

2> Docs created in previous 30 months;

3> Docs Created older than 30 months;

I then have to do some account id comaprisons across the 3 collections.

I have been using a view to get the collections e.g.

Set dc6Months=view6Months.GetAllDocumentsByKey(Cstr(doc.AccountID(0)))

However, this is proving SLOW as obviously the index on the view is causing an issue as it is rebuilt every time as the view selection formula uses @Adjust to select only the relevant documents.

Creation of the three collections takes about 25 seconds this way.

Any suggestions as to the most efficient way to create the three documentcollections?

This is all Notes Client UI functionality.

Thanks

2

Subject: Multiple documentCollections : Performance Issues & Most Efficient Way to do it?

Having views with @adjust in (I’m assuming you’re using @today or similar), will slow the db down so I’d get rid of those straight away anyway.

To build the collections I’d probably try an FTSearch. Same formula as your current view selection formulas, with the ID parameter added.

This is assuming your db is full text indexed. You could always use a dbsearch if not, but that would be slower

Dan

3

Subject: Multiple documentCollections : Performance Issues & Most Efficient Way to do it?

Hi Lemens,

why do you not create an all documents view, then you create a documentcollection with the getalldocumentsbykey method.

From this collection you pick out your 3 collections.

I think this will be faster than using the @adjust in the view selection formula.

Greetings

Raul

4

Subject: RE: Multiple documentCollections : Performance Issues & Most Efficient Way to do it?

But how would this work using GetAllDocumentsByKey?

The key is the AccountID in ALL cases. How, using this method do I just key the documents created in the last 6 months?

LS

5

Subject: RE: Multiple documentCollections : Performance Issues & Most Efficient Way to do it?

You get all the documents for a particular account (or all the view entries, if it happens that you can do all your work with view entries), and then you iterate thru that collection, looking at the created date of each document and reacting appropriately to it. If you use entries and a view sorted by (id then descending date), at some point you can stop scanning, and know the remaining docs in the collection are all old. If you really need to have three separate collections, you can put them into collections as you scan through – or you can just remember the index in the collection of the first entry for each date range. But without knowing more about what processing you’re doing with them, I can’t tell whether there’s even a need for that.

Bear in mind that you don’t have to use a NotesDocumentCollection to maintain a collection of documents; the List datatype is also handy for this, as are arrays, or you can define your own class for this purpose.