Performance issue - decision for Domino as appl. plattform

HCL Domino Domino Forum
1

I didn’t found any answer in Redbooks, Knowledge Base and notes.net.

Let’s discuss the following:

A Notes-application is used world-wide by about 2.000 users, 16 GB of data and 150.000 documents, 50% of the documents are updated daily (by backend-processes from SQL-DBs). Opening views is very slow (sometimes about 15 minutes) especially for views which are not used regular. No FT index on the database (because of regularly update of documents and size of FT index). The data consist of Notes-data and RTF-data like attachments. Only Notes-client access, separate Domino application-server.

What are the limits for a user-friendly Notes application: amount of views, documents from your experience?

What are the physical limitations in Domino 6 for views and documents?

Is Notes/Domino furthermore the right platform for this application?

Any, really any answers are welcome. We try to decide to use Domino furthermore as platform for this application.

2

Subject: Performance issue - decision for Domino as appl. plattform

A volatility rate of 75,000/day is pretty high.

How many servers is the application deployed to? Have you considered splitting into multiple NSFs to reduce individual load? What’s the geographic/network dispersal of the 2,000 users?

An FT index will generally help you, if it allows you to reduce the number of view indexes by permitting searching. You can also score some gains by eliminating rarely used views and replacing them with resortable columns in other views.

Whether Notes is the best tool depends in large part on what your alternatives are.

3

Subject: RE: Performance issue - decision for Domino as appl. plattform

The application is worlwide used and deployed over 4 servers.

The performance leak is accepted in the worldwilde use but not in the local area where users have normal bandwith.

Do you know how the index of the views influences the performance. We will think about eliminating rarely used views to get performance gains.

But for our decision can we enhance our application with further 100.000 documents? I read a lot about performance on IBM - United States and how applications are well designed. But nothing about the keys like ‘please avoid doing this and that’ e.g. not more than x views, not more than x documents, not more than x agents.

4

Subject: RE: Performance issue - decision for Domino as appl. plattform

Do you really need to modify so many documents in the sync with the relational database, or could you greatly reduce that figure with a more intelligent algorithm, e.g. the “Replicate Customers” agent from this download?

You gave a link to a performance resource page. Of all the things on that page, I suggest you pay the most attention to the redbook Performance Considerations for Domino Applications .

5

Subject: RE: Performance issue - decision for Domino as appl. plattform

Hi Robert,I’ve heard of some “rule-of-thumb” numbers in R4, but not for R6…

My personal opinion: 2000 users shouldn’t be the problem and 150.000 documents are not too much if you got the right machines. The 16GB of data are more than what I usually see, though.

Do you have any ideas what the load on your servers are from an OS point of view? CPU? Memory? HD?

As for a general approach to size your servers:

  • get as much disc I/O as you can (I mean data throughput rate here)

  • beg for some more disc I/O :wink:

  • get a good amount of memory

  • look for everything else that you might think is important

  • CPU is last

The bigger the DB’s are, the more docs you have, the more reads and - maybe even more important - the more writes and thus the neccessary view index rebuild - the higher the disc I/O is on your list.

Look for network storage, even mapped network drives, slowish hard discs in general.

Try to sort that I/O stuff first and then watch for your application.

Computed views might be a bottleneck, even worse when having content that is different everytime (i.e. columns contain current date/time calculations and stuff).

Move the computation into the document (QuerySave etc.).

Read in admin help: “Changing the temporary folder used for view rebuilds” → and watch your temp folder and log.nsf according to the tips and maybe tweak your notes.ini

Read in admin help: “Upgrading Domino 4.6 applications and Lotus Domino 6 requirements” and “Updating database indexes and views”

Watch your transaction logging: did you included the new R6 option to log the database views?

Another idea: if you periodically change lots of documents once a day - maybe run a view rebuild afterwards so the views are up to date in the morning?

OK - these are my first thoughts, hope it helps.

Regards,

Th.

6

Subject: RE: Performance issue - decision for Domino as appl. plattform

You’re not going to see hard limits on doc counts or views or agents because they don’t exist. More docs and more views and higher change rate in your data costs performance. Whether you can handle the performance is dependent on your app design, your server hardware, your bandwidth, and your user expectation.

I’d be mostly concern about your replication times. You’re describing potential data volatility in the area of > 2GB/day, which is a LOT of traffic across 4 servers. If your bandwidth is ok, then it’s just a question of having simple enough views, and few enough of them, to keep indexing reasonable with that change rate. How many you actually need depends on the business application.

It would be useful to answer the other questions I asked, by the way.

7

Subject: Performance issue - decision for Domino as appl. plattform

Are you transaction logging your view indexes?Using a separate disk/volume for view indexing?

For the sake of performance, you should eleminate as much views and/or sorted columns as possible. If you are sorting documents in the view on multiple items, consider adding a computed field with a combined sort key, so you can limit the sorting to one single column in the view.

The number of documents (150.000) is definitely not a problem, the size of the individual documents should not matter much for performance.

Since the amount of updates is that high, view indexing is your biggest issue. Any optimization on this (less views, less sorted columns, simpler view column formulas, less categorization, more disk iops) will be highly noticeable in the application.

Since you don’t mention any functional details of your setup, deciding whether domino is the best platform is hard.

cheers,

Bram

8

Subject: Performance issue - decision for Domino as appl. plattform

Are these 75,000 documents being updated throughout the day or in one large batch? If it’s one large batch then even if you’ve got 20+ views a moderately beefy ($5,000+) server should be able to update everything within an hour by Updall running in the middle of the night. Your trick in this case is keeping your database size low via compacts, etc.

If things are instead being updated throughout the day while users are accessing then you’ll need to really go after app design - trimming views, etc.

Also, are there complex readers fields on the docs? If you’ve got seriously complex readers fields (e.g. you have 15,000 users and each can only see their own 10 docs in the views) then the server CPU is a HUGE factor each time the view is opened or a @DbLookup is run against it.

9

Subject: Performance issue - decision for Domino as appl. plattform

If you have any column formulas, consider replacing them with items computed by the form, so as to reduce the load on the view indexer. I suppose that is even more important if the column is categorized. Eliminate as many views as you can.