Importing a file with 270'000 lines

HCL Domino Domino Forum
1

Hello

I’m trying to import a file (.txt) with 270’000 lines.

I’m using a agent with LotusScript. I’m reading the file with the following statments :

Open “\Saftp\Adresses_Fybad\BAN_SWIFT.txt” For Input As fileNum

Do While Not Eof(fileNum)

Line Input #fileNum, fileLine

Set doc = db.CreateDocument()

doc.Form=“Adresse”

doc.Code_BIC=Trim(Strrightback(Strleft(fileLine,“;”,0,3),“;”))

doc.Code_Branch=Trim(Strrightback(Strleft(fileLine,“;”,0,4),“;”))

…

Call doc.Save(True,True)

Loop

It works fine but it creates only 32’767 documents. It’ seems that the “Line input” does stop at this limit.

Is it true ?

How can I import this file without splitting it into 10 sub-files ?

Thanks for your help. :-)))

2

Subject: RE: Importing a file with 270’000 lines

It can be that You have declared the fileNum as Integer and 32767 is the limit of an integer value.

Use NotesStream to read large textfiles instead. Look in the Designer Help for this.

3

Subject: RE: Importing a file with 270’000 lines

You are right. FileNum was a integer. I changed it to Long and it’s ok. Thank you. I wish you a happy new year. :-)))

4

Subject: RE: Importing a file with 270’000 lines

That makes absolutely no sense to me. From the Domino Designer help for the Open statement:

fileNumber

An integer expression with a value between 1 and 255, inclusive. This number is associated with the file when you open the file. Other file-manipulation commands use this number to refer to the file.

Since it can only go up to 255 it should not matter what datatype it is. Furthermore, it is set when you open the file and does not get incremented. Unless you’re using it in some other way, it should not matter what datatype it is.

5

Subject: RE: Importing a file with 270’000 lines

In the loop, I had a counter (Integer) to count the line that the agent was reading. That was the problem. The only message I became was “Overflow”. And Kenneth give me a good way to the solution.

I paste you the code of the agent :

Sub Click(Source As Button)

Dim session As New NotesSession

Dim db As NotesDatabase

Dim doc As NotesDocument

Dim fileLine As String

Dim fileNum As Long

Dim totalAdresse As Long

Dim compteur As Long

Set db = session.CurrentDatabase

'donne le numéro de file libre pour l’importation

fileNum = Freefile

compteur = 0

'Ouvre le fichier FYBAD pour import

Open “\Saftp\Adresses_Fybad\BAN_SWIFT.txt” For Input As fileNum

Print “Début de l’importation des adresses SWIFT”

Do While Not Eof(fileNum)

'lecteur enregistrement

Line Input #fileNum, fileLine

compteur=compteur + 1

	

'création Adresse

Set doc = db.CreateDocument()

	

doc.Form="Adresse"

doc.Archive="Non"

	

'chargement par les valeurs lues

doc.Code_BIC=Trim(Strrightback(Strleft(fileLine,";",0,3),";"))

doc.Code_Branch=Trim(Strrightback(Strleft(fileLine,";",0,4),";"))

doc.Nom=Trim(Strrightback(Strleft(fileLine,";",0,5),";")) + Trim(Strrightback(Strleft(fileLine,";",0,6),";")) +Trim(Strrightback(Strleft(fileLine,";",0,7),";"))

doc.Localite= Trim(Strrightback(Strleft(fileLine,";",0,10),";"))

doc.Rue=Trim(Strrightback(Strleft(fileLine,";",0,14),";")) + Trim(Strrightback(Strleft(fileLine,";",0,15),";"))_

	+Trim(Strrightback(Strleft(fileLine,";",0,16),";"))+Trim(Strrightback(Strleft(fileLine,";",0,17),";"))

doc.NoPostal=Trim(Strrightback(Strleft(fileLine,";",0,18),";"))

doc.Ville=Trim(Strrightback(Strleft(fileLine,";",0,19),";"))+Trim(Strrightback(Strleft(fileLine,";",0,20),";"))+Trim(Strrightback(Strleft(fileLine,";",0,21),";"))

doc.LibPays=Trim(Strrightback(Strleft(fileLine,";",0,22),";"))+Trim(Strrightback(Strleft(fileLine,";",0,23),";"))

doc.IDAdr=Trim(Strrightback(Strleft(fileLine,";",0,35),";"))

doc.TypeAdr=Trim(Strrightback(Strleft(fileLine,";",0,11),";"))

	

Call doc.Save(True,True)

	

If compteur Mod 1000 = 0 Then

    Print "Nombre d'adresses importées = " + Cstr(compteur)

End If

Loop

Messagebox "ok importation terminée nb record lu : " + Cstr(compteur)

Print "ok importation terminée nb adresses créées : " + Cstr(compteur)

End Sub

6

Subject: RE: Importing a file with 270’000 lines

Yes, but beware - then long datatype has a limit as well; 2,147,483,647 to be exact. If Your file contains more lines than that - You will hit the roof once more. Therefore I suggest that You turn to NotesStream instead - it isn’t limited in size in any way and is more compatible with other platforms.