I’ve been asked to come up with a sizing estimate for Full Text Indexing our entire email environment. We currently have about 2500 databases consuming nearly 2Tb. I’ve looked at some databases and found that the fti for some can be about 10% of the total size whereas others can be 2/3/4 times to size of the database.
Does anyone know of a formula to use for calculating this?
Rob
Subject: Estimate for Full Text Index size
Hi,
Don’t know if you can know it for sure because it depends on what options are selected.
Here’s an old techote. The GTR engine changed in R5 but it could help:
Product Area:
Notes
Date:
97-10-23
Product Release:
Notes 4.x
Document #:
147752
Category:
Workstation/Desktop\Notes Client Functionality\Full Text Search
Descriptions of the Notes 4.x Full Text Index Settings
Problem:
In Notes 4.x, what are each of the Full Text Index settings available for a database?
Solution:
When you display the Properties (InfoBox) for database properties, the Full Text tab lists the Index settings. A description of each of these Index settings is provided below. (Note: The following information is taken directly from the Notes 4.x on-line Help.)
Case-sensitive index
When you create a case sensitive index, Notes indexes a word once for each case occurrence in the database. For example, “computer” and “Computer” both appear in the index as different words, so that searches can locate occurrences of one but not the other. Selecting this option increases the size of the full text index by 5 to 10 percent. This option is off by default.
Index attachments
This option lets you index the text in attachments and OLE objects. Only ASCII text is indexed and compressed attachments aren’t indexed. This option is off by default.
Index encrypted fields
This option lets you index text in encrypted fields. Selecting this option increases the size of an index according to the amount of text the encrypted fields contain.
Note the following security issues if you keep this option selected:
Any Notes user with access to the database can search for phrases within the encrypted fields without the encryption key. For example, the Employee form in the Personnel database contains the encrypted field Salary. With the full text index, any user can search on “50,000” and documents containing that figure are returned; however, the user cannot read the contents of the field without the encryption key.
The full text index file is unencrypted plain text, and anyone with access to the server can read the file. Encrypted text may now be unencrypted.
The encryption key is part of the server ID, and is active for all databases on that server. If you index a different database and do not deselect “Index Encrypted Fields,” any fields using that encryption key are compromised.
Exclude words in stop word file
A Stop word file contains words that you don’t want Notes to index – for example, a, an, be, and to. Selecting this option decreases the size of a full text index by 15 to 20 percent but also prevents the use of these words in queries. This option is off by default.
To determine which words to exclude, Notes refers to the text file DEFAULT.STP, located in the server program directory. This file contains a default list of words as well as the line [0-9]+ which tells Notes not to index numbers.
You can customize DEFAULT.STP using a text editor. Each excluded word must be on its own line followed by a carriage return. Keep in mind that the full text indexes for all databases on a server use DEFAULT.STP by default when the Stop word file option is selected.
You can create additional stop word files and customize them for specific databases. For example, if you index databases with medical topics, you can create a stop word file for those databases that includes words like medicine, doctor, and so on, that appear so frequently in documents that those words aren’t useful for searching. You must create an additional stop word file before you create the index that uses it. Then when you create the index, you select the customized stop word file. The file name must be eight characters or less and use the extension .STP, for example MEDICAL.STP. The stop word file must be located in the Notes program directory.
Stop word files don’t replicate; if you want replicas of a database to use the same customized stop word file for indexing, you must copy the customized stop word file to each server that contains a replica before you create an index.
Word breaks only
This option finds all matches for the search word, regardless of where the matches are located in the document. This is the default option.
Word, sentence, and paragraph
This option lets you apply operators to queries to find documents that contain two words in proximity to each other. For example, the following query finds documents in which “cat” and “mouse” are in the same paragraph:
cat paragraph mouse
Supporting Information:
Related Documents: