Hi
I am building an Notes client application that will hold millions of documents. The documents only have a few text fields ( between 10 and 20 fields), no richtect fields, subforms tables or other complex form objects. What must I consider ( or keep in mind) when dealing with performance of the database when millions of documents are involved?
It must also be possible to search the database in different ways ( keywords).
Regards
Subject: Performance of large Notes applications
Will you be using Reader fields to limit documents that people can see ? That can significantly effect view performance.
What is the frequency of updates to this data ? That can also effect views.
Subject: Performance of large Notes applications
With some experience trying to dance with a 2.5 million record database, first thing I’d consider is whether Domino is the vehicle you want here – particularly if it will be high transaction (changing constantly)
That said, keep the view indexes as simple as possible, talk with your backup people, pay for IBM support, and cancel any vacation plans you may have.
Subject: Performance of large Notes applications
Lando,
You definitely need to have some info about frequency of updates. If you’ll have approx 100/day, 1M/day, etc. You’ll also need to know how sophisticated the application will need to be in terms of things like workflow, reporting, etc., so that you know how hard you’ll be hitting lookup views and how many readers you’ll have.
You’ll also want to know approx how many concurrent sessions you’ll have, and how many total users.
At some point, you can start to model this on paper and then get a better sense of whether Domino is the right platform, and, if so, how lean you need to make your application in order to maintain solid performance.
There are some articles on these topics in LDD, but you probably need to get at least to a paper model before anything other folks write will feel useful.
Just my 2c,
regards,
raphael
Subject: RE: Performance of large Notes applications
Hi
The application will deal with about 500 users and about 100 concurrent. The creation of the documents are done in another database When the status of the created documents are set to ‘close’ the documents are transported at night to the archive database ( this is the databas I am talking about in my first posting). So the database is updated at night only. During the day the database is used for searching documents only.
Regards
Subject: RE: Performance of large Notes applications
Lando,
I’d probably try to make a single search view, maybe with a couple clickable columns. After you get your updates, make sure you run an agent (if necessary) to do a view.refresh so that the first user in the morning doesn’t get clobbered waiting for the view to open.
regards,
raphael
Subject: RE: Performance of large Notes applications
Based on what you described, it should not be a problem with R6 or R7 and fairly modern hardware. It has been a few years since I worked with an NSF with 2MM+ documents, it is definitely doable, especially since you are expecting very few updates during the daytime. The recommendations about optimizing your views and avoiding readernames fields should help a lot. I would think that a full-text index would improve search performance despite the size. The FTI will probably double the size of your database.
Subject: RE: Performance of large Notes applications
If possible (subject to the required business usage, of course), I’d NOT give the users access to any of the views.
Even with a single view, you’re looking at a long time for the view display to be rendered.
Rather, consider a “request” form type approach where the user enters criteria, and the search is performed and a list of suitable matches returned. Establish a maximum result-set to be returned.
A benefit of this approach is that if required the data storage may be moved elsewhere (SQL back-end for example) leaving the user interface intact.