Massive import to notes DB

Hi All

I’m looking for some guidance …

I have a need to import a large quantity of data via ODBC into a Notes database. I have written the code for this and it imports and produces around 32,000 notes documents. This is then replicated to another server in another office.

The routine runs each morning on the server and takes around 20 minutes to do a complete import.

I updated the code to compare what is on the SQL database and only import the changes. This again works well, but takes around one and a half hours to complete. It will work for several days and then starts to skip the odd piece of data (which is no good).

I did change the routine to delete all Notes documents before the import each day (so in effect I had a full import each day taking 20 mins) but this results in massive quantities of deletions stubs being retained.

Are there any smarter ways of doing this?

Thanks

Subject: How many fields? Do you have to handle deletions and additions of records?

Subject: RE: How many fields? Do you have to handle deletions and additions of records?

there are about 20 fields on each doucment. If rows are added to the SQL database, they need to be added to the Notes DB. Nothing is ever deleted so no need to worry about deletions of data

Subject: How did you handle your check while updating?

Something like this?flist = Split(“UniqueKey,Field1, …,Field20”, “,”)

existdoc = view.GetDocumentByKey(result.GetValue(flist(0)), True)

if existdoc is nothing then

set existdoc = db.CreateDocument

existdoc.Form = “Yours”

forall f1 in flist

call existdoc.ReplaceItemValue(f1, result.GetValue(f1))

end forall

else

forall f2 in flist

v1 = existdoc.GetFirstItem(f2)

v2 = result.GetValue(f2)

if v1.Text <> v2 then

call existdoc.ReplaceItemValue(f2, v2)

end if

end forall

end if

call existdoc.Save(True, False) 'Note that this will not do a save if no fields were modified in the else loop above.